Resource finder overview

From Wiki for iCub and Friends
Jump to: navigation, search

Starting from YARP > 2.3.23 and iCub > 1.1.13 this documentation has been revised and moved to the yarp official documentation: http://wiki.icub.org/yarpdoc/yarp_resource_finder_tutorials.html

This page is now obsolete, it is here only for future reference.

Authors: Lorenzo Natale and Paul Fitzpatrick

Introduction

The problem we address here is how to organize files used by modules. These files can be configuration/ini files or resource files (for example data files or images used by a module). We will refer in general to them as resource files.

In most cases running a module as:

mymodule --file ./iniFile.ini

seems appropriate. However for a large distribution of executables this becomes a problem because you cannot always assume you know the working directory of your module.

In the RobotCub software repository we organize parameters and files used by modules (in short we call them 'resources') in a directory structure in $ICUB_ROOT/app. The advantage of this approach is that all resources are localized in a single directory structure that can be completely relocated (moved) if/when needed (e.g. you want to move this structure to your own home directory).

We want to decouple the location of the configuration files and that of the executable. When developing in fact we can assume we know the executable working directory (although when running using "yarp run" when cannot control the process working directory, at least not yet). When executable are "installed" (as in a binary distribution) executable and configuration files might end up being moved.

We would like to organize resource files in a directory structure. This structure can be in $ICUB_ROOT/app, but we organize things so that the whole tree can be easily relocated.

Resource files are accessed through a YARP object, the ResourceFinder (or RF). To locate a certain file the programmer asks the RF using a "key". The RF searches the file system to identify the requested file and, if successful, it returns a full path to it; the search is performed using the rules that are specified when the RF is first created or configured. Parameters are passed to the RF through the command line or a file. The RF is not only responsible for locating files but can also decode parameters passed to the module and make them available as a key-value list.

Let's start from an example

Let's write a module called 'random_motion' that performs random movements of a single joints. The module is generic in that it can control any of the available joints of one of the limbs of either the robot or the simulator.

The module receives the following parameters:

--robot name
--part  robotpart
--joint j

an runs, for example, as:

random_motion --robot icub --part head --joint 0

We can use the ResourceFinder to manage the parameters to this module. So far this is nothing new, but we will see that it has some advantages:

ResourceFinder rf;
rf.setVerbose();
rf.configure("ICUB_ROOT", argc, argv);

This creates an instance of the ResourceFinder, and configures it from data from the command line. For now we don't bother about the string "ICUB_ROOT". It is explained in more details in the documentation.

Now we can query the value of the parameters by doing:

ConstString robotName=rf.find("robot").asString();
ConstString partName=rf.find("part").asString();
int joint=rf.find("joint").asInt();

In this respect the ResourceFinder behaves exactly like a Searchable (if you know what this is). However there are important differences.

First of all, if you prefer you can create a txt file that stores all parameters to a module:

random.ini:
robot icub
part head
joint 0

and simply run your module as:

random_motion --from random.ini

The ResourceFinder will be initialed using the content of the file "random.ini". Subsequent queries to the rf will return the values stored in the file.

We can make this even simpler if we specify a default configuration file:

rf.setDefaultConfigFile("random.ini");

so that you can just run:

random_motion

without parameters.

This is handy and quite flexible. You could have a set of initialization files for your module and get different behaviors by running the same executable with different parameters. However, you could also organize files in separate directories and just point your module to different directories every time you want to change initialization files. The names of these files do not need to change, what changes is their location in the file system.

In the RobotCub repository we follow exactly this approach. Configuration files are stored in directories in $ICUB_ROOT/app each representing different ways to instantiate your module (or collection of modules). We call this the "initialization context" of your module. You easily switch behavior by switching the "context" in which your module (the ResourceFinder) looks for config files.

In our example we can create different contexts from which our simple module will run. In $ICUB_ROOT/app/tutorials we have created two directories called randomMotion and randomMotionSim. Each directory contains a similar file, called "random.ini", with slightly different parameters.

You can instantiate the randomMotion module using the file "random.ini" from the different contexts by running:

random_motion --context tutorials/randomMotion/conf

or

random_motion --context tutorials/randomMotionSim/conf

to achieve different behaviors of the module depending on the content of the file app/tutorials/randomMotion/random.ini or /app/tutorials/randomMotionSim/conf/random.ini

If you like, you can suggest a default value for the context, just add the following line:

rf.setDefaultContext("tutorials/randomMotion/conf");

Now running:

random_motion

without parameters will result in random_motion being configured with the content of app/tutorials/randomMotion/conf/random.ini.

ResourceFinder Overview

The RF is key-value list which contains parameters of a module. These parameters can be specified in the command line as:

 mymodule --key1 value1 --key2 value2

or in an "initialization file" as:

 mymodule --from file.ini

in which file.ini is:

 key1 value1
 key2 value2
 ...

The RF works as a searchable. The method find(key) (or check) returns a value starting from a key.

First you need to configure the RF:

 rf.configure("ICUB_ROOT", argc, argv);

The first string specifies a "key" to locate the file which points to the "resource search path policy file" (or "policy file"). The policy file describes/specifies the policy used to search for resource files. For example it could specify that resource files are searched in $ICUB_ROOT/app. This opens up the possibility to modify the behavior of the RF later on, in case we decided we want to store resource files in different locations (for example an app directory inside the user's HOME). The policy file itself is searched stating from the "key", using certain rules, see below.

You can query the RF in this way:

 ConstString value1=rf.find("key1").asString();
 int value2=rf.find("key2").asInt();

to retrieve the corresponding values of the parameters specified either from the command line or from the initialization file (file.ini).

RF looks for initialization files following a certain policy. We skip now the details of how this policy works (see below). In short the RF will look in a directory called "initialization context", that is specified from the command line.

 mymodule --from file.ini --context myModule/conf

makes RF search in $ICUB_ROOT/app/myModule/conf

If you like you can specify a default initialization file:

rf.setDefaultConfigFile("file.ini");

so that you don't have to repeat it every time you run your module:

mymodule --context myModule/conf

you can specify a default context:

rf.setDefaultContext("myModule/conf");

so that you can run your module simply as:

mymodule

If needed you can switch the context from which your module is initialized by running it as:

mymodule --context myModule2/conf

assuming "file.ini" exists in $ICUB_ROOT/app/myModule2

Resource Finder for localizing files

The RF works also as file finder. The method findFile(key) picks the value corresponding to key and interprets it as a filename. The RF does its best to locate the file named value and if successful returns a string which contains the full path to the file. Localization of resource files follows the same rules described for the "initialization file" above.

Another example

Suppose our module now also needs a file (this could be another initialization file or a binary file

such as an image or a file containing data in binary format).

Suppose you store parameters in a file called mymodule.ini:

 mymodule.ini:
 robot icub
 part  head
 data  data.bin

mymodule will now use the RF to get parameters and locate data.bin.

First we need to instantiate and configure the RF.

Example code in main.cpp

 ...
 ResourceFinder rf;
 rf.configure("ICUB_ROOT", argc, argv);
 rf.setDefaultConfigFile("mymodule.ini");

As usual we can query the rf:

 ConstString robotName=rf.find("robot").asString();
 ConstString partName=rf.find("part").asString();
 ...

More importantly we can also get a full path pointing to data.bin:

ConstString iniFile=rf.findFile("inifile").asString;
if (iniFile=="")
   // print error
else 
   // open file

When we run:

 mymodule --context context

the RF will search both mymodule.ini and data.bin in $ICUB_ROOT/app/context.

Rules to locate the policy file

What are the rules followed by the ResourceFinder to localize resources?

Given:

rf.configure("KEY", argc, argv)

rf will:

  • if the environment variable KEY exists, search for $KEY/KEY.ini
  • if this fails, in Linux search in /etc/KEY.ini

Order of preference when searching for resource files

When searching for files the RF will:

  • search if the file exists in the local directory (local here means with respect to the process working directory)
  • check relative the directory where the default config file is (specified with --from), this is useful when the config file is specified with relative path (as in: --from ../../conf/file.ini)
  • check relative to a context specified on the command line (--context)
  • check relative to a context specified in the code (setDefaultContext)

Contexts are interpreted as specified by the policy file.

All this can be overridden by the policy file.