Difference between revisions of "ResourceFinder"

From Wiki for iCub and Friends
Jump to navigation Jump to search
Line 10: Line 10:
* each type of files must be placed inside a specific sub-directory. Current allowed sub-directories are listed [[# Data directories contents | here]].
* each type of files must be placed inside a specific sub-directory. Current allowed sub-directories are listed [[# Data directories contents | here]].


A full description of how the directories and environment variables used by the ResoureFinder is too detailed for most users. We provide here a short description of the files layout of typical configurations for Windows, Linux and MacOs.
  TODO: add typical file layout for installation/build tree of Windows/MacOS and Linux Debian.
  TODO: move detailed description in specific page





Revision as of 00:42, 19 November 2013

These instructions only apply to YARP > 2.3.23 and iCub > 1.1.13 software versions

YARP's ResourceFinder allows to find and access data and configuration files at run-time. It is used both by YARP's own executables (to locate generic configuration files and "description files" that allow to manage YARP-based programs) and by user modules (to locate specific, "context-based" configuration files and data). YARP has an OS-dependent set of default locations where to look for files, and a set of rules that define the search priorities. The default locations can be overridden or extended by modifying the environment variables that YARP relies on (see this section). The key points to keep in mind are:

  • files can be "spread" over the system, inside user-specific or installation directories
  • installation directories are generally non-writable, but users can (and should) customize or add files inside their own "home" directories
  • external packages can install files so that YARP can find them (see this section)
  • each type of files must be placed inside a specific sub-directory. Current allowed sub-directories are listed here.

A full description of how the directories and environment variables used by the ResoureFinder is too detailed for most users. We provide here a short description of the files layout of typical configurations for Windows, Linux and MacOs.

 TODO: add typical file layout for installation/build tree of Windows/MacOS and Linux Debian.
 TODO: move detailed description in specific page


Directories used by ResourceFinder

Broadly speaking, there are three ways in which the ResourceFinder operates to locate files:

  • looking in the current working directory
  • according to a policy
  • performing a system-wide search according to the following environment variables
YARP_CONFIG_HOME
Location where user config files are stored <path to a directory>

Unless the user wants to test some specific setting, this environment variable should be unset

YARP_DATA_HOME
Location where user data files are stored <path to a directory>

Unless the user wants to test some specific setting, this environment variable should be unset

YARP_CONFIG_DIRS
Locations where system administrator data and config files are stored <list of path separated by columns>

These directories can be used by the system administrator to modify some specific setting for the system

YARP_DATA_DIRS
Locations where installed data and config files are stored <list of path separated by columns>.

These are usually the directories where the installers put the data files.

If a user wants to use uninstalled packages, he should modify this environment variable


When searching for files and directories, the ResourceFinder looks inside these directories in the above order, so that files modified by the user take precedence over installed ones.

The default value for these variables is dependent on the operating system.


Linux

Default values for Linux are based on the XDG Base Directory Specification (see http://standards.freedesktop.org/basedir-spec/basedir-spec-0.8.html ). They rely on XDG environment variables, which default to:

$XDG_DATA_HOME
-> $HOME/.local/share/
$XDG_CONFIG_HOME
-> $HOME/.config/
$XDG_DATA_DIRS
-> /usr/local/share/:/usr/share/
$XDG_CONFIG_DIRS
-> /etc/xdg/

Default values for Yarp are:

$YARP_DATA_HOME
-> $XDG_DATA_HOME + /yarp/
$YARP_CONFIG_HOME
-> $XDG_CONFIG_HOME + /yarp/
$YARP_DATA_DIRS
-> $XDG_DATA_DIRS + /yarp/ (appended to each dir)
$YARP_CONFIG_DIRS
-> $XDG_CONFIG_DIRS + /yarp/ (appended to each dir) [NOTE: /etc/yarp is used as default instead of /etc/xdg/yarp ]

Windows

The following environment variables (with their respective default values) are available on windows:

%YARP_DIR%
-> C:\Program Files\robotology\YARP_2.X.XX or C:\Program Files (x86)\robotology\YARP_2.X.XX (Set by the installer or by the user)
%HOMEPATH%
-> C:\Users\{username} or C:\Documents and Settings\{username} (User home)
%APPDATA%
-> %HOMEPATH%\AppData\Roaming (Windows 7) or %HOMEPATH%\Application Data (Windows XP) (Location where applications should store their data by default)
%ALLUSERSPROFILE%
-> C:\ProgramData (Windows Vista) or C:\Documents and Settings\All Users (Windows XP) (Location of the "All Users" or "Common" profile folder)


Default values for Yarp are therefore:

%YARP_DATA_HOME%
-> %APPDATA%\yarp
%YARP_CONFIG_HOME%
-> %APPDATA%\yarp\config\
%YARP_DATA_DIRS%
-> %YARP_DIR%\share\yarp\
%YARP_CONFIG_DIRS%
-> %ALLUSERSPROFILE%\yarp\

Mac OsX

On Mac, the following environment variable is used:

$HOME
-> /Users/<username>

Default values for Yarp are therefore:

$YARP_DATA_HOME
-> $HOME/Library/Application Support/yarp
$YARP_CONFIG_HOME
-> $HOME/Library/Preferences/yarp
$YARP_DATA_DIRS
-> /usr/local/share/yarp:/usr/share/yarp
$YARP_CONFIG_DIRS
-> /etc/yarp:Library/Preferences/yarp


Configuration Files

Config files are INI files that contain parameters used by Yarp's own executables.

a) User $YARP_CONFIG_HOME

b) Sysadmin $YARP_CONFIG_DIRS + /config/

c) Installed ($YARP_DATA_DIRS && path.d) + /config/

path.d

path.d is a special folder used by yarp to provide a portable way to install 3rd party applications in separate folders (see here). It is searched for by the Resource Finder in the same locations as "Installed" configuration files (c) before searching for any "installed" resource. Each file inside this path.d folder contains the path to the "data directory" of the corresponding installed package; this path is therefore appended to the list of paths that make up the Installed locations list (c).

Data directories contents

The following directories should be present inside each of the $YARP_DATA_HOME, $YARP_CONFIG_DIRS and $YARP_DATA_DIRS directories so that their content can be found by the Resource Finder

For any of the above-mentioned directories <dir>, the possible locations are a) User $YARP_DATA_HOME + /<dir>/

b) Sysadmin $YARP_CONFIG_DIRS + /<dir>/ [should not be normally needed]

c) Installed ($YARP_DATA_DIRS && path.d) + /<dir>/

Contexts and Robots files

The contexts folders contain multiple sub-folders. Each subfolder represents a context, i.e., a collection of configuration files and data that may be used to configure modules at runtime and read/write any type of files. When the resource finder is configured with a specific context-name (either a default, hard-coded one, or provided as command line parameter to the module), contexts/<context-name> is added to the search path in which the initial configuration file and any additional files are sought. The same <context-name> can appear in user and installed contexts directories, which means that files associated to a context can actually be located in multiple places (i.e., there are both user files and installed files associated to the same context; user files "mask" installed files with the same name)

The robots folders contain multiple sub-folders as well. Each subfolder has the name of a robot, and contains robot-specific files. The ResourceFinder adds robots/$YARP_ROBOT_NAME (default value for this environment variable: default) to the search path, before the context-specific paths.

Plugins Manifest Files

Plugin manifest files are INI files that describe which class implements the plugin, the library it is included in, and how to find such library. These files are needed by the yarpdev executable.


Application deployment files

Applications, modules and resources folders contain XML files that describe programs (modules), the resources available to run the programs (resources), and collections of interconnected modules (applications). The yarpmanager and gyarpmanager executables allow to run and monitor applications, while the gyarpbuilder executables allows to design new ones.

Templates are XML files with a .template extension. Typical templates are applications that need to be configured (i.e., assigning the node where each module should be run) before they can be executed.

Generation of the search path

When looking for an initial configuration file, the ResourceFinder will look inside the following directories (in order):

  • current working directory
  • robot directory
    • user robot directory
    • installed robot directory (searches also for path.d directory)
  • (if a context has been specified) context directory
    • user context directory
    • installed context directory
  • $YARP_CONFIG_HOME
  • $YARP_DATA_HOME
  • $YARP_CONFIG_DIRS
  • $YARP_DATA_DIRS (and path.d )

Subsequent searches for files may follow different routes if the initial configuration file was found relative to the "current working directory" ("local" behaviour). In this case, the search path becomes:

  • current working directory
  • initial configuration file directory (might be different from previous one if the initial configuration file is provided with a relative path)
  • robot directory
    • user robot directory
    • installed robot directory (searches also for path.d directory)
  • $YARP_CONFIG_HOME
  • $YARP_DATA_HOME
  • $YARP_CONFIG_DIRS
  • $YARP_DATA_DIRS (and path.d )


3rd party packages

Installed packages

Yarp exports the following CMake variables:

  • YARP_INSTALL_PREFIX
  • YARP_CONFIG_INSTALL_DIR
  • YARP_PLUGIN_MANIFESTS_INSTALL_DIR
  • YARP_MODULES_INSTALL_DIR
  • YARP_APPLICATIONS_INSTALL_DIR
  • YARP_TEMPLATES_INSTALL_DIR
  • YARP_APPLICATIONS_TEMPLATES_INSTALL_DIR
  • YARP_MODULES_TEMPLATES_INSTALL_DIR
  • YARP_CONTEXTS_INSTALL_DIR
  • YARP_ROBOTS_INSTALL_DIR

Using the correct CMake variables when installing files will ensure that those files will be in the correct place on every platform.

Also non-root installation is supported, but both yarp and 3rd party software will require to be configured with the same $PREFIX, and the environment variables will require to be tweaked accordingly.

Uninstalled packages

Paths to uninstalled data directories can be appended to the $YARP_DATA_DIRS environment variable. Please note that Yarp will consider these packages as installed; local user data should be put inside $YARP_DATA_HOME


Packages that want to keep their stuff in their own data directory

Packages that want to install their files in their own directory, should install a .ini file in $YARP_PREFIX/yarp/config/path.d . This can be done using the yarp_configure_external_installation macro provided in <YARP_SOURCE>/conf/YarpInstallationHelpers.cmake.