YARP Cleanup

From Wiki for iCub and Friends
Revision as of 17:05, 14 November 2012 by Daniele.Domenichelli@iit.it (talk | contribs) (Versioning)
Jump to: navigation, search

NOTE: THIS PAGE IS A DRAFT --Daniele.Domenichelli@iit.it 15:17, 14 November 2012 (CET)



This is a list of issues that are required and/or desirable in order to cleanup YARP, and to make it easier to package, and appealing for other projects.


General

  • [Required] Create yarp-devel mailing list and mail to the rc-hackers mailing list

CMake 2.6.0

  • [Required] At the moment YARP is not compiling with cmake 2.6.0, but officially we are supporting it. Moreover internally it searches for cmake 2.6.4. My suggestion is to procede in this way:
    • Fix the actual issue with cmake 2.6.0.
    • Release 2.3.20.
    • Write a mail to the mailing list about the new policies for supported debian/ubuntu releases.
    • Add a wiki page for supported versions.
    • Check if the build machines for the new policy are available and eventually add them.
    • Create 2.3 stable branch and start development of 2.4 series in trunk.
    • Bump cmake version to 2.6.4.
    • Cleanup and remove all workarounds for cmake 2.6.0.
    • Ensure that everywhere 2.6.4 is required (in all code and documentation).
    • Ensure that after those changes, YARP actually builds with cmake 2.6.4 (check the dashboard)

FindXXX

  • [Recommended] Fix all the FindXXX.cmake files dropping compatibility, according to CMake_Modules_Guidelines
  • [Recommended] Remove ACE4YARP
  • [Recommended] Use FeatureSummary and replace the YARP_USE_XXX and YARP_SKIP_ACE with HAVE_XXX
  • [Optional] Investigate if FindGtk shipped with cmake can be used instead of our homemade version, and eventually fix it upstream.
  • [Required] Mail to the mailing lists about those changes.

Dependency Issues

If you link libYARP_os, you don't need to link directly to ACE, unless you are using it directly in your code. So the aim here is to fix wherever libraries not needed are linked, in order to reduce the dependencies needed for debian packages (in this case you don't need the libace-dev package containing the libACE.so file)

  • [Recommended] Use LINK_PRIVATE in order not to link unneeded packages.
  • [Recommended] Create and export variables YARP_OS_LIBRARY, and use them instead of YARP_LIBRARIES
  • [Recommended] Ensure that all packages are linked internally using only the required packages and not using YARP_LIBRARIES
  • [Required] Mail to the mailing lists about those changes.

Plugin framework

Current plugin framework has several issues with dependencies. The solution would be to have plugins that are loaded dynamically and not linked statically (RUNTIME option is still not a solution because if one .so file is deleted, the library crashes, therefore the dependencies are still the same).

  • [Recommended] Rewrite plugin framework that does not require to link everything from every carrier and device. libYARP_init can be dropped if the plugin framework is implemented properly.
  • [Suggested] Remove all the non-free plugin stuff and put it in another repository. (This would be necessary to have yarp in the free section of debian repository, and the nvidia/cuda plugins in the contrib section, otherwise the whole yarp should be in contrib). When plugins will be loaded at runtime, there will be no problem in keeping them separate.
  • [Required] Mail to the mailing lists about those changes.

Extern libraries

Debian/Ubuntu

There are a few issues with extern libraries in Debian/Ubuntu:

  1. TinyXML is required both by YARP and iCub, and the version installed with squeeze is too old.
  2. OpenCV requires several flags, and not all of them are supported in all debian/ubuntu versions.
  3. GtkDataboxMM is packaged only for Debian Testing.

At the moment I don't know how to solve these issues, the ideal solution would be not to have them in YARP/iCub and have only the FindXXX.cmake for them. But to arrive to this goal,

  • [Recommended] We should have apt repositories with dependencies with the debian/ubuntu versions that we support. In those repositories we should have the required versions of the external libraries.
  • [Recommended] We should work a lot together with packagers for Debian/Ubuntu to get the versions and the build flags that we need in the upstream packages.

Windows

Windows is a problem by itself, but on this issue, it is relatively safe, since we are already shipping binaries for most of the external libraries.

  • [Suggested] We should ship external packages together with GtkMM and the other dependencies.
  • [Suggested] We should investigate if it is possible to use CoApp for building YARP packages and its dependencies (website seems to be dead at the moment)

If one day we manage to do so, we could remove all the extern stuff and keep that in some external repository. But for now I don't know what's the best way to deal with it.

General

  • [Proposal] Move external dependencies in a "robocub-support" repository (to be installed before YARP and ICUB).

Most of this stuff is not required for packaging for linux distro, because it's already there. Moreover Debian policies require to remove all non-free stuff and all binary blobs, therefore the tarball released might have to be repackaged downstream. Handling this in an external repository might make things easier. From this repository we can check if a package is not installed, and eventually build and install a private version of that package.

API/ABI and Versioning

Application Programming Interface

A "stable" API guarantees that code written for a version of Yarp should compile with a newer version. A library that guarantees API stability is usually preferable when writing some software, because this implies that you won't have to update your software when the library is updated and/or you won't have to keep different branches working with different versions of the library.

Application Binary Interface

A "stable" ABI guarantees that code compiled for a version of Yarp should run with a newer version of the library installed. The advantage is that when packaging newer version of the library, if there are programs built using the previous version, they don't need to be recompiled.

SONAME

From Debian Policy Manual

Every time the shared library ABI changes in a way that may break binaries linked against older versions of the shared library, the SONAME of the library and the corresponding name for the binary package containing the runtime shared library should change. Normally, this means the SONAME should change any time an interface is removed from the shared library or the signature of an interface (the number of parameters or the types of parameters that it takes, for example) is changed. This practice is vital to allowing clean upgrades from older versions of the package and clean transitions between the old ABI and new ABI without having to upgrade every affected package simultaneously.
The SONAME and binary package name need not, and indeed normally should not, change if new interfaces are added but none are removed or changed, since this will not break binaries linked against the old shared library.

To check the SONAME you can run objdump -p /opt/iit/lib/libYARP_OS.so | grep SONAME

References

Versioning

Versioning should reflect the changes in the code I suggest to adopt a strict policy in numbering, and to use:

  • YARP_VERSION_MAJOR
    Fixed (2) until there is some very big change
    YARP_VERSION_MINOR
    Increase if API is changed
    YARP_VERSION_PATCH
    Increase if API is not changed, reset to 0 when YARP_VERSION_MINOR is increased.
    SONUMBER
    rename the libraries to libYARP2_xxx.so and restart counting from 0 and increase it properly. This number should not be "global", but depending on the library.

Ideally, when increasing YARP_VERSION_MINOR number, the API should be stable (no methods removed, but marked as deprecated instead) and all those deprecated methods should be removed when increasing the YARP_VERSION_MAJOR number

When doing a minor release, a branch should be started as well, and bugfix should be applied both to master and stable branches.

A policy often adopted is to use the MINOR number to distinguish between stable and unstable releases. Another policy adopted is to set in the master branch the MINOR number very big (i.e. 60) and increase it when there is an incompatible change, or a new feature (i.e. the role of YARP_VERSION_TWEAK)

TODO

  • [Suggested] Constify methods and pass const references whenever possible before the 2.4.0 release. For example
    • virtual bool open(Searchable& config) should probably be virtual bool open(const Searchable& config)
    • virtual Bottle& findGroup(const char *key) should probably be virtual const Bottle& findGroup(const char *key) const
  • [Suggested] Starting with the 2.4 series we should run API and ABI checker tools before every release to ensure that API and ABI are stable during the release cycle.
    • Methods should be marked as deprecated instead of modifying/removing them, and eventually removed in a major release, after a reasonable amount of time (Example).
  • [Suggested] Do not install private headers (impl)
    • People should not rely on those headers that should contain pure implementative details.
    • If someone requires parts of them, we should consider moving that part to the public headers.
  • [Check] visibility=hidden should be used by default and only the public methods should be exported.
  • [Check] Exported classes should not contain private members and methods, since those are implementation details (see 1 and 2).
  • [Recommended] Adopt a strict versioning policy.

Other Optional Cleanup

  • [If we have time] Replace all tabs with spaces and remove trailing spaces
  • [If we have time] Coding style (brackets, etc.)
  • [If we have time] Headers should should not require other headers before in order to compile
  • [If we have time] Headers should contain the minimum amount of other headers, using forward declarations where possible.
  • [If we have time] Remove packaging stuff from the main repository.
  • [If we have time] Not all the classes/features in YARP are covered by unit tests, we should add them
  • [In my dreams] We should migrate to Git