Salience

The correct title of this article is salience. The initial letter is shown capitalized due to technical restrictions.

Authors: Paul Fitzpatrick, Jonas Ruesch

Find interesting things in an image stream. This module applies preattentive filters to an image stream in order to extract a "salience" estimate for every part of the currently visible scene. There are currently filters for motion, saturated color, skin tone, and faces.

Webpage for this module: http://eris.liralab.it/iCub/dox/html/group__icub__salience.html

Back to iCub YARP module specifications

Compiler & Linker Dependencies

Include files

iCub/SalienceModule.h>
iCub/Salience.h>        // Filter interface
iCub/XYSalience.h>      // for each filter
iCub/RemoteSalience.h>  // for remoting

Libraries

YARP
OpenCV
STL
fastfilt (iCub library)

Run-time Dependencies

Module arguments

If desired, command line arguments can be moved to a configuration file by specifying --file NAME.ini. This is useful for complicated configurations. Here is an example ini file:

    name             /salience
    filter           group
    filterName       group
    numBlurPasses    3
    drawSaliencePeak 1
    thresholdSalience 10.0

    [subfilter ruddy]
    filterName ruddy
    weight 0.7

    [subfilter color]
    filterName color
    weight 0.7

    [subfilter emd]
    filterName EMD
    weight 0.2
    alpha 0.28
    threshold 10.0
    blurInput 1
    blurOutput 1
    constrain 1
    constrainValue  255.0
    scale      1.0

The basic structure is some overall configuration options, then a set of "stanzas" for filters.

Ports accessed

Ports that are assumed to exist prior to instantiation of the module (i.e. some other module must create them)

There needs to be a wide-field-of-view-camera input available. On iCub, this is called:

   /icub/cam/left

Ports created

Ports that are instantiated by the module and are then available for other modules to use (using yarp connect)

Under normal use, the following ports are created. The port names are just examples (controlled by options):

Input ports

    /salience/view     This port expects to receive a stream of wide-field-of-view
                       images from a camera.

    /salience/conf     Salience weights etc. can be reconfigured by sending  input to this port.

output ports

    /salience/view      This port generates a modified copy of the stream of images received, 
                        to give an impression of the salience attached to different parts of the scene.  
                        This is intended for human monitoring purposes only.

    /salience/map       This is a floating point array of the same size as the input
                        image, with higher values at a point representing a 
                        higher salience level.

Input data files

Haar wavelet cascade filter coefficients for face detector.

Output data files

None

Configuration files

See #Module arguments. An example configuration file can be found here iCub/conf/chica_salience.ini.

User interface mechanism

Run-time modification of module parameters

qsaliencegui      Graphical user interface for the salience module
                  Weighting of each filter can be adjusted by using a slider.

Remote access provided through remoteEgosphere

User interface currently through module terminal, e.g. set/get xyValue 0.5

OS on which the module was developed

Windows, Linux

OS on which the module was tested

Windows, Linux

Operating system dependencies

None known

Example instantiation of the module

Run & port connection commands

REM Copyright (C) 2007 Jonas Ruesch
REM CopyPolicy: Released under the terms of the GNU GPL v2.0.

REM Simple example instantiation for the camCalib module.
REM You need to adapt values between <> according to your environment in order to make this work.
REM The module configuration is loaded from a file specified by the --file option. To create this file
REM have a look at the default terminal output of the module. A module reports it's option keys 
REM and default values on startup. Some modules require an additional option --group which specifies
REM the name of the configuration group within the configuration file (specify without []).
REM We assume you compiled in release mode.

:: Start module
start <ICUB_DIR>\bin\release\salience.exe --name /iCub/salience --file <x:/path/to/config.ini>

:: Start viewers 
start yarpview --RefreshTime 29 --name /view/salience

:: Connect input ports
yarp wait /iCub/salience/view
yarp connect </camera> /iCub/salience/view

:: Connect  viewer
yarp wait /view%NAME_SALIENCE%
yarp connect /iCub/salience/view /view/salience

See also iCub/scripts/startChicaAttentionSelection.bat

iCub Capabilities

iCub capability code

C6   Construct sensorimotor maps & cross-modal maps
C8   Exploratory, curiousity-driven, action
C10  Crawl and navigate based on local landmarks and ego-centric representations
C11  Saccadic direction of gaze towards salient events (visual, auditory, tactile)

Refer to iCub capabilities for a complete list of iCub capabilities.

Other Yarp modules required to effect this capability

camCalib
controlGaze
attentionSelection

Example instantiation

See:

iCub/scripts/startChicaEyeRight.bat
iCub/scripts/startChicaSalienceRight.bat
iCub/scripts/startChicaEgosphere.bat
iCub/scripts/startChicaAttentionSelection.bat
iCub/scripts/startChicaGazeControl.bat

Back to iCub YARP module specifications