Difference between revisions of "YARP ResourceFinder"

From Wiki for iCub and Friends
Jump to: navigation, search
(YARP Base Directory Specification)
(XDG Base Directory Specification)
Line 5: Line 5:
  
  
== XDG Base Directory Specification ==
+
= Directory used by ResourceFinder =
 +
 
 +
 
 +
== XDG Base Directory Specification (Linux) ==
  
  
Line 28: Line 31:
 
; $XDG_RUNTIME_DIR:
 
; $XDG_RUNTIME_DIR:
 
   -> /tmp/something (? see specs)
 
   -> /tmp/something (? see specs)
 
 
  
 
== Windows Directories ==
 
== Windows Directories ==

Revision as of 22:12, 27 February 2013

NOTE: THIS PAGE IS A DRAFT --Daniele.Domenichelli@iit.it 18:17, 27 February 2013 (CET)



Directory used by ResourceFinder

XDG Base Directory Specification (Linux)

See http://standards.freedesktop.org/basedir-spec/basedir-spec-0.8.html

$XDG_DATA_HOME
  -> $HOME/.local/share/
$XDG_CONFIG_HOME
  -> $HOME/.config/
$XDG_DATA_DIRS
  -> /usr/local/share/:/usr/share/
$XDG_CONFIG_DIRS
  -> /etc/xdg/
I don't like this at all, and I think it is not used almost by any program. I suggest that we use /etc/ as default instead
if $XDG_CONFIG_DIRS is not set. Anyway in this document we assume to use the correct default --Daniele.Domenichelli@iit.it 18:28, 27 February 2013 (CET)

Not used?

$XDG_CACHE_HOME
  -> $HOME/.cache/
$XDG_RUNTIME_DIR
 -> /tmp/something (? see specs)

Windows Directories

 %YARP_DIR%
Set by the installer or by the user
 %HOMEPATH%
User home (C:\Users\{username} or C:\Documents and Settings\{username})
 %APPDATA%
Location where applications should store their data by default.
             -> %HOMEPATH%\AppData\Roaming (Win7)
             -> %HOMEPATH%\Application Data (WinXP)
 %ALLUSERSPROFILE%
Location of the "All Users" or "Common" profile folder.
             -> C:\ProgramData (WinVista)
             -> C:\Documents and Settings\All Users (Win XP)

Not used?

 %LOCALAPPDATA%
Location of temporary files of applications. (Since WinVista)


Mac OsX Directories

?


YARP Base Directory Specification

Linux

 $YARP_DATA_HOME     -> $XDG_DATA_HOME    +  /yarp/
 $YARP_CONFIG_HOME   -> $XDG_CONFIG_HOME  +  /yarp/
 $YARP_DATA_DIRS     -> $XDG_DATA_DIRS    +  /yarp/ (appended to each dir)
 $YARP_CONFIG_DIRS   -> $XDG_CONFIG_DIRS  +  /yarp/ (appended to each dir)

Windows

 $YARP_DATA_HOME     -> %APPDATA%\yarp
 $YARP_CONFIG_HOME   -> %APPDATA%\yarp\config\
 $YARP_DATA_DIRS     -> %YARP_DIR%\share\yarp\
 $YARP_CONFIG_DIRS   -> %ALLUSERSPROFILE%\yarp\

Mac OsX

 $YARP_DATA_HOME     -> ?
 $YARP_CONFIG_HOME   -> ?
 $YARP_DATA_DIRS     -> ?
 $YARP_CONFIG_DIRS   -> ?


$YARP_DATA_HOME and $YARP_CONFIG_HOME are one directories $YARP_DATA_DIRS and $YARP_CONFIG_DIRS are a list of directories


============================= Configuration Files ==============================



Config files are INI files.

a) User         $YARP_CONFIG_HOME
        # LINUX     -> $HOME/.config/yarp/
        # WINDOWS   -> %APPDATA%\yarp\config
        # MAC       -> ?

b) Sysadmin     $YARP_CONFIG_DIRS  +  /config/
        # LINUX      -> /etc/xdg/yarp/config/
        # WINDOWS    -> %ALLUSERSPROFILE%\yarp\config\
        # MAC        -> ?

c) Installed    $YARP_DATA_DIRS  +  /config/
        # LINUX      -> /usr/local/share/yarp/config/:/usr/share/yarp/config/
        # WINDOWS    -> %YARP_DIR%\share\yarp\config\
        # MAC        -> ?



A config files are searched in a), b), and c), in this order.
All of the files found should be considered, but for a specific setting, only
the first value is considered. This allows to have all settings installed in c),
the sysadmin can overwrite some of them in b), and the user can tweak the
behaviour in a)
Command line parameters should always take precedence over config files,

CMake will export a ${YARP_CONFIG_INSTALL_DIR} variable, that should be used in
3rd party software to install config files in the right directory



================================== Data Files ==================================



Data files are all the architecture independent files  (text files, images,
audio, video, etc) that are installed by yarp or by some 3rd party software)

Generic data files are located as follow

a) User         $YARP_DATA_HOME
        # LINUX     -> $HOME/.local/share/yarp/
        # WINDOWS   -> %APPDATA%\yarp\
        # MAC       -> ?

b) Sysadmin     $YARP_CONFIG_DIRS
        # LINUX     -> /etc/xdg/yarp/
        # WINDOWS   -> %ALLUSERSPROFILE%\yarp\
        # MAC       -> ?

c) Installed    $YARP_DATA_DIRS
        # LINUX     -> /usr/local/share/yarp/:/usr/share/yarp/
        # WINDOWS   -> %YARP_DIR%\share\yarp\
        # MAC       -> ?


Depending on the type of data file, the resource finder should be able to have
different behaviour (in pseudo-code a possible implementation of the methods)
The ResourceManager should have methods for these different behaviour.


* FIND ALL BY TYPE
  * Find all the resources of a specified type, and show them to the user as
    different files, perhaps in a treeview (for example template files)
        list<string> RF::findAll(string resourceType, RF::Installed|RF::Sysadmin|RF::User);
  * Find all the resources of a specified type and for each unique name, return
    only one choosing in order User, Sysadmin and finally Installed
    If more than one file with the specified name was found in the same level
    (i.e. Installed) we should be able to choose if show the first, the last, or
    all of them.
        list<string> RF::findAll(string resourceType, RF::Installed|RF::Sysadmin|RF::User|RF::One|RF::First);
        list<string> RF::findAll(string resourceType, RF::Installed|RF::Sysadmin|RF::User|RF::One|RF::Last);
        list<string> RF::findAll(string resourceType, RF::Installed|RF::Sysadmin|RF::User|RF::One|RF::All);
  * Find all the resources of a specified type in the "Installed" directories.
    If more than one file with the specified name was found we should be able
    to choose if show the first, the last, or all of them
        list<string> RF::findAll(string resourceType, RF::Installed|RF::First);
        list<string> RF::findAll(string resourceType, RF::Installed|RF::Last);
        list<string> RF::findAll(string resourceType, RF::Installed|RF::All);
  * Find all the resources of a specified type in the "Sysadmin" directories
        list<string> RF::findAll(string resourceType, RF::Sysadmin|RF::First);
        list<string> RF::findAll(string resourceType, RF::Sysadmin|RF::Last);
        list<string> RF::findAll(string resourceType, RF::Sysadmin|RF::All);
  * Find all the resources of a specified type in the "User" directory
        list<string> RF::findAll(string resourceType, RF::User);


* FIND ALL BY TYPE AND NAME
  * Find all the resources of a specified type and name and return all of them
        list<string> RF::findAll(string resourceType, string name, RF::Installed|RF::Sysadmin|RF::User);
  * Find all the resources of a specified type and name in the "Installed"
        directories and return all of them.
        list<string> RF::findAll(string resourceType, string name, RF::Installed);
  * Find all the resources of a specified type and name in the "Installed"
        directories and return all of them.
        list<string> RF::findAll(string resourceType, string name, RF::Sysadmin);


* FIND ONE BY TYPE AND NAME
  * Find all the resources of a specified type and name and return only the top
    one (in this order: User, Sysadmin, and finally Installed)
    If more than one file with the specified name was found in the same level
    (i.e. Installed) we should be able to choose if show the first or the last.
        string RF::find(string resourceType, string name, RF::Installed|RF::Sysadmin|RF::User|RF::First);
        string RF::find(string resourceType, string name, RF::Installed|RF::Sysadmin|RF::User|RF::Last);
  * Find all the resources of a specified type and name in the "Installed"
    directories and return only one of them.
    If more than one file with the specified name was found in the same level
    (i.e. Installed) we should be able to choose if show the first or the last.
        string RF::find(string resourceType, string name, RF::Installed|RF::First);
        string RF::find(string resourceType, string name, RF::Installed|RF::Last);
  * Find all the resources of a specified type and name in the "Sysadmin"
    directories and return only one of them.
    If more than one file with the specified name was found in the same level
    (i.e. Installed) we should be able to choose if show the first or the last.
        string RF::find(string resourceType, string name, RF::Installed|RF::First);
        string RF::find(string resourceType, string name, RF::Installed|RF::Last);
  * Find the resource of a specified type and name in the "User" directory
        string RF::find(string resourceType, string name, RF::User);


* MERGE ALL FILES WITH GIVEN TYPE AND NAME
  * Find all of them and merge them (for example for config files, a setting
    should be searched first in the User file, if not found in the Sysadmin
    file, and finally if not found also in the Installed file)
    If more than one file with the specified name was found in the same level
    (i.e. Installed) we should be able to choose whether priority should be
    given to the first found or to the last found
        Searchable RF::readConfig(string resourceType, string name, RF::First);
        Searchable RF::readConfig(string resourceType, string name, RF::Last);


I strongly recommend that we avoid allowing recursive search, or it will make
a lot harder everything. It doesn't mean that we don't allow to have
subdirectories, but each subdirectory will contain one resource type. For
example the templates folder will have an application directory: application
template types will be "templates/applications"




CMake will export a ${YARP_DATA_INSTALL_DIR} variable, that should be used in
3rd party software to install config files in the right directory



============================ Plugins Manifest Files ============================



Plugin manifest files are INI files.

a) User         $YARP_DATA_HOME  +  /plugins/
        # LINUX     -> $HOME/.local/share/yarp/plugins/
        # WINDOWS   -> %APPDATA%\yarp\plugins\
        # MAC       -> ?

b) Sysadmin     $YARP_CONFIG_DIRS  +  /plugins/
        # LINUX     -> /etc/xdg/yarp/plugins/
        # WINDOWS   -> %ALLUSERSPROFILE%\yarp\plugins\
        # MAC       -> ?

c) Installed    $YARP_DATA_DIRS  +  /plugins/
        # LINUX     -> /usr/local/share/yarp/plugins/:/usr/share/yarp/plugins/
        # WINDOWS   -> %YARP_DIR%\share\yarp\plugins\
        # MAC       -> ?

Plugins should include the absolute path to the library to open that includes
the class implementing the plugin.
While this is not a problem for installed plugins, it might be a problem for
uninstalled stuff. Therefore we might need something


CMake will export a ${YARP_PLUGIN_MANIFESTS_INSTALL_DIR} variable, that should
be used in 3rd party software to install config files in the right directory



================================= Module Files =================================



Module files are XML files.
They are usually associated to one executable

a) User         $YARP_DATA_HOME  +  /modules/
        # LINUX     -> $HOME/.local/share/yarp/modules/
        # WINDOWS   -> %APPDATA%\yarp\modules\
        # MAC       -> ?

b) Sysadmin     $YARP_CONFIG_DIRS  +  /modules/
        # LINUX     -> /etc/xdg/yarp/modules/
        # WINDOWS   -> %ALLUSERSPROFILE%\yarp\modules\
        # MAC       -> ?

c) Installed    $YARP_DATA_DIRS  +  /modules/
        # LINUX     -> /usr/local/share/yarp/modules/:/usr/share/yarp/modules/
        # WINDOWS   -> %YARP_DIR%\share\yarp\modules\
        # MAC       -> ?


User data is writable, the other directories no, so when a user creates a new
module, opens a template or a module in a read only folder, it will be saved
in the User folder


Configuration for modules is set with parameters passed in the application, and
with configuration files. Configuration files are handled in the same way, as
described before.


CMake will export a ${YARP_MODULES_INSTALL_DIR} variable, that should be used in
3rd party software to install config files in the right directory



============================== Application Files ===============================



Application files are XML files.

a) User         $YARP_DATA_HOME  +  /applications/
        # LINUX     -> $HOME/.local/share/yarp/applications/
        # WINDOWS   -> %APPDATA%\yarp\applications\
        # MAC       -> ?

b) Sysadmin     $YARP_CONFIG_DIRS  +  /applications/
        # LINUX     -> /etc/xdg/yarp/applications/
        # WINDOWS   -> %ALLUSERSPROFILE%\yarp\applications\
        # MAC       -> ?

c) Installed    $YARP_DATA_DIRS  +  /applications/
        # LINUX     -> /usr/local/share/yarp/applications/:/usr/share/yarp/applications/
        # WINDOWS   -> %YARP_DIR%\share\yarp\applications\
        # MAC       -> ?


User data is writable, the other directories no, so when a user creates a new
application, opens a template or a application in a read only folder, it will
be saved in the User folder


Even though applications are usually shipped as templates, CMake will export a
${YARP_APPLICATIONS_INSTALL_DIR} variable, that should be used in 3rd party
software to install config files in the right directory.




================================ Resource Files ================================



Resource files are INI files. They are not installed with yarp or with 3rd party
software, but are configured by the user or by the system administrator.

The name "Resource" is in my opinion ambiguous, so I strongly recommend to try
to find a better name for these files


a) User         $YARP_DATA_HOME  +  /resources/
        # LINUX     -> $HOME/.local/share/yarp/resources/
        # WINDOWS   -> %APPDATA%\yarp\resources\
        # MAC       -> ?

b) Sysadmin     $YARP_CONFIG_DIRS  +  /resources/
        # LINUX     -> /etc/xdg/yarp/resources/
        # WINDOWS   -> %ALLUSERSPROFILE%\yarp\resources\
        # MAC       -> ?


Installed directories is not needed, because this kind of files is not supposed
to be installed by external packages.

For the same reason also CMake variable is not needed.



================================ Template Files ================================



Template files are usually XML files. They are not supposed to be modified by
the user, but both the user and the sysadmin can add their own templates.

Template base dir:

a) User         $YARP_DATA_HOME  +  /templates/
        # LINUX     -> $HOME/.local/share/yarp/templates
        # WINDOWS   -> %APPDATA%\yarp\templates
        # MAC       -> ?

b) Sysadmin     $YARP_CONFIG_DIRS  +  /templates/
        # LINUX     -> /etc/xdg/yarp/templates/
        # WINDOWS   -> %ALLUSERSPROFILE%\yarp\templates\
        # MAC       -> ?

c) Installed    $YARP_DATA_DIRS  +  /templates/
        # LINUX     -> /usr/local/share/yarp/templates/:/usr/share/yarp/templates/
        # WINDOWS   -> %YARP_DIR%\share\yarp\templates
        # MAC       -> ?

Each template is installed in a subdirectory (applications, modules, resources,
etc.), depending on its type.

Template files with the same name are not exclusive, that means that if an
application template file "foo.xml" is found in a), b), and c), the user should
see all of them when creating a new application from template


CMake will export the ${YARP_TEMPLATE_INSTALL_DIR} (generic),
${YARP_APPLICATION_TEMPLATES_INSTALL_DIR} and
${YARP_MODULE_TEMPLATES_INSTALL_DIR} (specific for application and modules)
variables, that should be used in 3rd party software to install template files
in the right directory.
Resource templates should be installed only by YARP, therefore there is no need
for a CMake variable



================================ Advanced Uses =================================



(Note that in the following examples, the XDG_ variables are used on linux only)


* User wants to use a separate folder to test something, maintaining multiple
  configurations of the same application

  # For ALL applications  -> Change $XDG_CONFIG_HOME
  # For YARP only         -> Change $YARP_CONFIG_HOME


* User wants to avoid using sysadmin settings

  # For ALL applications  -> Change $XDG_CONFIG_DIRS (config files are searched
                             in the yarp subdirectories)
  # For YARP only         -> Change $YARP_CONFIG_DIRS


* User wants to add settings from a package not installed

  # For ALL applications  -> Append values to $XDG_CONFIG_DIRS (i.e.
                             "/etc/xdg:/new/folder") (config files are searched
                             in the yarp subdirectories)
  # For YARP only         -> Append values to $YARP_CONFIG_DIRS (i.e.
                             "/etc/xdg/yarp:/new/folder/yarp")


* User wants to add data from a package not installed

  # For ALL applications  -> Append values to $XDG_DATA_DIRS (i.e.
                             "/etc/xdg:/new/folder") (config files are searched
                             in the yarp subdirectories)
  # For YARP only         -> Append values to $YARP_DATA_DIRS (i.e.
                             "/etc/xdg/yarp:/new/folder/yarp")


* User wants to keep 2 separate installations of yarp

  # For ALL applications  -> Change $XDG_DATA_DIRS (config files are searched in
                             the yarp/config subdirectories)
  # For YARP only         -> Change $YARP_DATA_DIRS (config files are searched
                             in the config subdirectories)



============================== 3rd party packages ==============================



* Installed packages

  Using the correct CMake variables when installing files will ensure that those
  files will be in the correct place on every platform.

  Also non-root installation is supported, but both yarp and 3rd party software
  will require to be configured with the same $PREFIX, and the environment
  variables will require to be tweaked accordingly.


* Uninstalled packages can be appended to the $YARP_DATA_DIRS environment
  variable


  # LINUX
      -> $YARP_DATA_DIRS -> $XDG_DATA_DIRS/yarp/:$ICUB_ROOT
    User can set the variable in his .bashrc, sysadmin can set the variable
    system wide by adding a .sh file in /etc/profile.d/
  # WINDOWS
      ->  $YARP_DATA_DIRS -> %YARP_DIR%\share\yarp\:%ICUB_DIR%\share\yarp\
    Edit system variables (user and system)
  # MAC
    ?


* Packages that want to keep their stuff in their own data directory

If we want to allow packages to install their files in their own directory, we
could use the /etc/yarp/path.d directory as we discussed about.
Nonetheless, instead of using the /etc directory, since we are dealing with
files installed by another package, I suggest to use /usr/share/yarp/path.d
instead.