Plugins manifests and config files
Discussion page to better understand how to organize files like plugins ini files, manifests for modules and applications and configuration files.
The idea is to come up wiith a solution that works for the following situations:
- YARP as independent library
- YARP and iCub coexisting together
In addition we support the usual view in which software can be installed or compiled locally.
Configuration files that need to be found one way or another
There are a few low-level config files used by YARP:
- yarp_namespace.conf -- current yarp namespace
- yarp.conf -- this stores the network location of a name server, and what type of name server it is
- alternate .conf files are used if yarp is in a namespace other than the default one.
These config files predate the existence of ResourceFinder.
YARP plugins that are loaded at runtime rather than compiled in may come with config files, currently named like this:
This is new stuff, unstable.
- yarpmanager.ini for gyarpbuilder
- iCub applications have lots of config files, bootstrapped from an ICUB_ROOT.ini file
- iCub plugins will maybe have config files? (not vital for device plugins; vital for carrier plugins)
- The name server may have a yarpserver.conf file
- yarpview may have one/many .conf files?
- ... what else? ...
The plugin system need to fins *.ini files. These files could be in a single or multiple directories.
When YARP is installed, plugins ini files will be installed in a known directory.
When YARP is compiled locally plugins ini files will be in the binary tree.
Other packages i.e. iCub can contribute other plugins. They may be installed in a known location or compiled locally.
Note: not talking about dll or .so, these follow the rules of the os.
From the current yarp tutorial:
YARP is configured to aggregate plugin.ini files in a directory called etc/yarp/plugins. The ini files are collected in this directory irrespective of whether the corresponding plugins are compiled or not. To add plugins not included with YARP, an ini file may be placed in etc/yarp/plugins.
Question: do you need env variables to get this behavior? I would say no but sentence below seems to contradict.
YARP looks for etc/yarp/plugins using a yarp::os::ResourceFinder. If you don't already use configuration files with YARP, one way to get going would be to create a file in your home directory called .yarp/plugins.ini (or yarp\plugins.ini on Windows) containing:
style capability default_capability /path/to/your/yarp/build
and then set a YARP_POLICY environment variable to plugins (matching the name of the ini file you placed in the .yarp/yarp directory).
- No, env variables not needed, that was just an example --pf
Question: wouldn't it be better if plugins could use a specific policy (i.e. YARP_PLUGINS); in this way there would be no need for setting YARP_POLICY when yarp is installed (assuming this is the case). Other possibilities:
- put plugins ini files in the same directories in which we put configuration files (for YARP and for iCub); each "framework" will have its own set of configuration files and plugins (and maybe manifest), following rules according to their own policies (YARP_POLICY and ICUB_ROOT).
- similar to the above but no need to have separate policy for plugins, if we add to the policy file (i.e. YARP_POLICY.ini/ICUB_ROOT.ini) a specific entry for plugins
Comment from pf: Constraint on solution: YARP will need to be able to find all plugins, since it is responsible for loading them when needed (am I correct in this assumption? --pf). How should YARP find iCub plugins?
Comment from lorenzo: Assumption correct. I thought doing
export YARP_POLICY=ICUB_ROOT yarpdev --device ...
was the idea. Not optimal but sort of ok.
We use the ResourceFinder and a policy file. iCub checks:
- /etc/ICUB_ROOT.ini --> ICUB_ROOT.ini points to /usr/share/iCub or something like that.
- $ICUB_ROOT/ICUB_ROOT.ini --> ICUB_ROOT.ini points to "app" locally in $ICUB_ROOT.
So far this has been OK to support installations 1) and local compilation 2).
In YARP yarpscope and gyarpmanager/gyarpbuilder need also config files. YARP_POLICY is checked. This allows for YARP users to have their own configuration files. I guess as specified in /etc/YARP_POLICY.ini or $YARP_POLICY/YARP_POLICY.ini.
YARP and iCub can coexist. If YARP_POLICY is equal to an existing environment variable, i.e. ICUB_ROOT then iCub and YARP can share the same config files.
gyarpbuilder needs manifest files for modules. At the moment it uses YARP_POLICY to find ymanager.ini; this contains a list of directories to be parsed looking for xml manifest files. Normally iCub users use $ICUB_ROOT/app/yarpmanager.ini.
This works for local compilation. However in the case in which we install YARP and iCub, no environment variables are set and gyarpmanager goes to search /etc/YARP_POLICY.ini.
- Policy files are in /etc; however there was previous suggestion from Daniele that they should go to /etc/yarp or /etc/iCub. This is a good idea especially if we add more files to /etc/yarp.
- It looks like using YARP_POLICY and ICUB_ROOT allows handling almost all situations. This does not seem to work well for gyarpbuilder on the other hand, since this module needs to access multiple directories at the same time. Since this problem seems only relevant for gyarpbuilder we could also find a specific solution.
At the moment gyarpbuilder has it own yarpmanager.ini configuration file. This could go in ~/.yarp, /etc/yarp or be located with YARP_POLICY.
To make iCub and YARP coexist:
- Users modify manually yarpmanager.ini to add $ICUB_ROOT/app
- iCub provides its own yarpmanager.ini; users modify it to add $YARP_ROOT/YARP_DIR/path/to/modules and point YARP_POLICY to ICUB_ROOT.
I prefer 1 which seems less work. The only problem is that fixing yarpmanager.ini automatically at install time is not something easy to do (need to know where either ICUB_ROOT or YARP_ROOT are).