Configuration and resource files

From Wiki for iCub and Friends
Jump to: navigation, search

Starting from YARP > 2.3.23 and iCub > 1.1.13 this documentation has been revised and moved to the yarp official documentation: http://wiki.icub.org/yarpdoc/yarp_resource_finder_tutorials.html

This page is now obsolete, it is here only for future reference.

Authors: Lorenzo Natale and Paul Fitzpatrick

Introduction

The problem we are discussing here is how to organize files used by modules. These files can be configuration/ini files or resource files (for example data files or images used by a module). We will refer in general to them as resource files.

The problem originates from the fact that we want to decouple the location of the configuration files and that of the executable. When developing in fact we can assume we know the executable working directory (although when running using "yarp run" when cannot control the process working directory, at least not yet). When executable are "installed" (as in a binary distribution) executable and configuration files might end up being moved.

We would like to organize resource files in a directory structure. This structure can be in $ICUB_ROOT/app, but we organize things so that the whole tree can be easily relocated.

Resource files are accessed through a YARP object, the ResourceFinder (or RF). To locate a certain file the programmer asks the RF using a "key". The RF searches the file system to identify the requested file and, if successful, it returns a full path to it; the search is performed using the rules that are specified when the RF is first created or configured. Parameters are passed to the RF through the command line or a file. The RF is not only responsible for locating files but can also decode parameters passed to the module and make them available as a key-value list.

ResourceFinder Overview

The ResouceFinder should become the gateway to access parameters and resource files in a module. The RF is key-value list which contains parameters of a module. These parameters can be specified in the command line as:

 mymodule --key1 value1 --key2 value2

or in an "initialization file" as:

 mymodule --from file.ini

in which file.ini is:

 key1 value1
 key2 value2
 ...

The RF works as a searchable. The method find(key) (or check) returns a value starting from a key.

First you need to configure the RF:

 rf.configure("ICUB_ROOT", argc, argv);

The first string specifies a "key" to locate the file which points to the "resource search path policy file" (or "policy file"). The policy file describes/specifies the policy used to search for resource files. For example it could specify that resource files are searched in $ICUB_ROOT/app. This opens up the possibility to modify the behavior of the RF later on, in case we decided we want to store resource files in different locations (for example an app directory inside the user's HOME). The policy file itself is searched stating from the "key", using certain rules, see below.

You can query the RF in this way:

 ConstString value1=rf.find("key1").asString();
 int value2=rf.find("key2").asInt();

to retrieve the corresponding values of the parameters specified either from the command line or from the initialization file (file.ini).

RF looks for initialization files following a certain policy. We skip now the details of how this policy works (see below). In short the RF will look in a directory called "initialization context", that is specified from the command line.

 mymodule --from file.ini --context myModule

makes RF search in $ICUB_ROOT/app/myModule

If you like you can specify a default initialization file:

rf.setDefaultConfigFile("file.ini");

so that you don't have to repeat it every time you run your module:

mymodule --context myModule

you can specify a default context:

rf.setDefaultContext("myModule");

so that you can run your module simply as:

mymodule

If needed you can switch the context from which your module is initialized by running it as:

mymodule --context myModule2

assuming "file.ini" exists in $ICUB_ROOT/app/myModul2

Resource Finder for localizing files

The RF works also as file finder. The method findFile(key) picks the value corresponding to key and interprets it as a filename. The RF does its best to locate the file named value and if successful returns a string which contains the full path to the file. Localization of resource files follows the same rules described for the "initialization file" above.

ResourceFinder API

 // Get the value of the specified key, then search for a file with that name.
 // If the key has not been given a value, try searching for a file with the
 // key's name.
 ConstString findFile(const char *keyName);
 // Set a default value of a key
 bool setDefault(const char *keyName, const char *keyValue);
 // Set a name which, in conjunction with the policy, defines a default directory 
 // to search for files.  Should be called before configure()
 bool setDefaultContext(const char *contextName);
 // Load the policy, and apply command line options.  Options can modify
 // the policy, change key values, and change the search path
 bool configure(const char *policyName, int argc, char *argv[]);

Rules to locate the policy file

Given:

rf.configure("KEY", argc, argv)

rf will:

  • if the environment variable KEY exists, search for $KEY/KEY.ini
  • if this fails, in Linux search in /etc/KEY.ini

Order of preference when searching for resource files

When searching for files the RF will:

  • search if the file exists in the local directory (local here means with respect to the process working directory)
  • check relative the directory where the default config file is (specified with --from), this is useful when the config file is specified with relative path (as in: --from ../../conf/file.ini)
  • check relative to a context specified on the command line (--context)
  • check relative to a context specified in the code (setDefaultContext)

According to the ICUB_ROOT policy it will also:

  • new since July 2009: if the environment variable ICUB_ROBOTNAME exists, check robot specific directory (app/$ICUB_ROBOTNAME/conf)
  • check default conf directory in app/default/conf

Let's make an example

Suppose you have a module that needs the following parameters to run: robot and part name plus a file.

You can run the module as:

 mymodule --robot icub --part head --inifile moreparameters.ini

mymodule will use a RF to decode all these parameters and find moreparameters.ini.

First we need to instantiate and configure the RF.

Example code in main.cpp

 ...
 ResourceFinder rf;
 rf.configure("ICUB_ROOT", argc, argv);
 ...

this configures ResouceFinder using information in $ICUB_ROOT/ICUB_ROOT.ini.

We specify a default context:

 rf.setDefaultContext("mymodule");

to be appended to the "app" directory for searching resources in $ICUB_ROOT/app/mymodule. A user will be able to overwrite the context at the command line with the '--context' parameter.

Passing argc and argv to the RF allows us to pass the parameters --robot, --part and --inifile, so that later we can query the RF to get the corresponding value:

 const char *robotName=rf.find("robot").asString().c_str();
 const char *partName=rf.find("part").asString().c_str();
 ...

When we need to locate (and open) inifile we use the findFile method:

 const char *iniFile=rf.findFile("inifile");
 if (iniFile==0)
   // print error
 else 
   // open file

Optionally we can write a short mymodule.ini file embeds all parameters to the module:

 mymodule.ini:
 robot icub
 part  head
 inifile moreparameters.ini

and run the module simply as:

 myModule --from mymodule.ini

Test case examples

Note: this section contains examples that useful to discuss implementation details, it is not meant to be used for documentation.

Consider the takeOverTheWorld module.

The parameters of takeOverTheWorld are:

 --robot : name of the robot
 --weapon: weapon config file
 --resource: a jpg image of the weapon in use

Example:

 takeOverTheWorld --robot icub --weapon weapon.ini --resource /resources/sharks.jpg

These parameters can be placed in a file, e.g. takeOver.ini.

In $ICUB_ROOT/app there are two contexts:

 takeOverWithSharks
 takeOverWithSeaBass

which configure the module in slightly different ways.

The same structure is replicated locally in: In $ICUB_ROOT/takeOverTheWorld/conf/example1 and $ICUB_ROOT/takeOverTheWorld/conf/example2

This is a list of example with the expected behavior.

Testing rule: use default context

  $ICUB_ROOT/src/takeOverTheWorld/takeOverTheWorld --from takeOver.ini

should resolve:

  $ICUB_ROOT/app/takeOverWithSharks/takeOver.ini
  $ICUB_ROOT/app/takeOverWithSharks/weapon.ini
  $ICUB_ROOT/app/takeOverWithSharks/resources/sharks.jpg

Currently: works, default context applies to takeOver.ini

Testing rule: use specific context, give priority to local files in this context

  $ICUB_ROOT/src/takeOverTheWorld/takeOverTheWorld --context takeOverWithSeaBass --from takeOver.ini

should resolve:

  $ICUB_ROOT/app/takeOverWithSeaBass/takeOver.ini
  $ICUB_ROOT/app/takeOverWithSeaBass/weapon.ini
  $ICUB_ROOT/app/takeOverWithSeaBass/resources/seabass.jpg

Currently: works.

Testing rule: use specific from file (respect "tab" rule), give priority to local files

  $ICUB_ROOT/src/takeOverTheWorld/takeOverTheWorld --from ./conf/example1/takeOver.ini

should resolve:

  $ICUB_ROOT/src/takeOverTheWorld/conf/example1/takeOver.ini
  $ICUB_ROOT/src/takeOverTheWorld/conf/example1/weapon.ini
  $ICUB_ROOT/src/takeOverTheWorld/conf/example1/resources/sharks.jpg

Currently: works.

  $ICUB_ROOT/src/takeOverTheWorld/takeOverTheWorld --from ./conf/example2/takeOver.ini

should resolve:

  $ICUB_ROOT/src/takeOverTheWorld/conf/example2/takeOver.ini
  $ICUB_ROOT/src/takeOverTheWorld/conf/example2/weapon.ini
  $ICUB_ROOT/src/takeOverTheWorld/conf/example2/resources/seabass.jpg

Currently: works.

  $ICUB_ROOT/src/takeOverTheWorld/takeOverTheWorld --from ./conf/example3/takeOver.ini

should resolve:

  $ICUB_ROOT/src/takeOverTheWorld/conf/example3/takeOver.ini
  $ICUB_ROOT/src/takeOverTheWorld/conf/example3/weapon.ini
  and fail to locate $ICUB_ROOT/src/takeOverTheWorld/conf/example3/resources/sharks.jpg (file is missing)

Currently: works, but weird uses weapon.ini and sharks.jpg from default context.

Testing rule: use default file and default context

  $ICUB_ROOT/src/takeOverTheWorld/takeOverTheWorld

should resolve:

  $ICUB_ROOT/app/takeOverWithSharks/takeOver.ini
  $ICUB_ROOT/app/takeOverWithSharks/weapon.ini
  $ICUB_ROOT/app/takeOverWithSharks/resources/sharks.jpg

Currently: works.

Testing rule: use default file and specific context

  $ICUB_ROOT/src/takeOverTheWorld/takeOverTheWorld --context takeOverWithSeaBass

should resolve:

  $ICUB_ROOT/app/takeOverWithSeaBass/takeOver.ini
  $ICUB_ROOT/app/takeOverWithSeaBass/weapon.ini
  $ICUB_ROOT/app/takeOverWithSeaBass/resources/seabass.jpg

Currently: works.