Salience
- The correct title of this article is salience. The initial letter is shown capitalized due to technical restrictions.
Authors: Paul Fitzpatrick, Jonas Ruesch
Find interesting things in an image stream. This module applies preattentive filters to an image stream in order to extract a "salience" estimate for every part of the currently visible scene. There are currently filters for motion, saturated color, skin tone, and faces.
Webpage for this module: http://eris.liralab.it/iCub/dox/html/group__icub__salience.html
Back to iCub YARP module specifications
Compiler & Linker Dependencies
Include files
iCub/SalienceModule.h> iCub/Salience.h> // Filter interface iCub/XYSalience.h> // for each filter iCub/RemoteSalience.h> // for remoting
Libraries
YARP OpenCV STL fastfilt (iCub library)
Run-time Dependencies
Module arguments
If desired, command line arguments can be moved to a configuration file by specifying --file NAME.ini
.
This is useful for complicated configurations. Here is an example ini file:
name /salience filter group filterName group numBlurPasses 3 drawSaliencePeak 1 thresholdSalience 10.0 [subfilter ruddy] filterName ruddy weight 0.7 [subfilter color] filterName color weight 0.7 [subfilter emd] filterName EMD weight 0.2 alpha 0.28 threshold 10.0 blurInput 1 blurOutput 1 constrain 1 constrainValue 255.0 scale 1.0
The basic structure is some overall configuration options, then a set of "stanzas" for filters.
Ports accessed
Ports that are assumed to exist prior to instantiation of the module (i.e. some other module must create them)
There needs to be a wide-field-of-view-camera input available. On iCub, this is called:
/icub/cam/left
Ports created
Ports that are instantiated by the module and are then available for other modules to use (using yarp connect)
Under normal use, the following ports are created. The port names are just examples (controlled by options):
Input ports
/salience/view This port expects to receive a stream of wide-field-of-view images from a camera. /salience/conf Salience weights etc. can be reconfigured by sending input to this port.
output ports
/salience/view This port generates a modified copy of the stream of images received, to give an impression of the salience attached to different parts of the scene. This is intended for human monitoring purposes only. /salience/map This is a floating point array of the same size as the input image, with higher values at a point representing a higher salience level.
Input data files
Haar wavelet cascade filter coefficients for face detector.
Output data files
None
Configuration files
See #Module arguments. An example configuration file can be found here iCub/conf/chica_salience.ini
.
User interface mechanism
Run-time modification of module parameters
qsaliencegui Graphical user interface for the salience module Weighting of each filter can be adjusted by using a slider.
Remote access provided through remoteEgosphere
User interface currently through module terminal, e.g. set/get xyValue 0.5
OS on which the module was developed
Windows, Linux
OS on which the module was tested
Windows, Linux
Operating system dependencies
None known
Example instantiation of the module
Run & port connection commands
REM Copyright (C) 2007 Jonas Ruesch REM CopyPolicy: Released under the terms of the GNU GPL v2.0. REM Simple example instantiation for the camCalib module. REM You need to adapt values between <> according to your environment in order to make this work. REM The module configuration is loaded from a file specified by the --file option. To create this file REM have a look at the default terminal output of the module. A module reports it's option keys REM and default values on startup. Some modules require an additional option --group which specifies REM the name of the configuration group within the configuration file (specify without []). REM We assume you compiled in release mode. :: Start module start <ICUB_DIR>\bin\release\salience.exe --name /iCub/salience --file <x:/path/to/config.ini> :: Start viewers start yarpview --RefreshTime 29 --name /view/salience :: Connect input ports yarp wait /iCub/salience/view yarp connect </camera> /iCub/salience/view :: Connect viewer yarp wait /view%NAME_SALIENCE% yarp connect /iCub/salience/view /view/salience
See also iCub/scripts/startChicaAttentionSelection.bat
iCub Capabilities
iCub capability code
C6 Construct sensorimotor maps & cross-modal maps C8 Exploratory, curiousity-driven, action C10 Crawl and navigate based on local landmarks and ego-centric representations C11 Saccadic direction of gaze towards salient events (visual, auditory, tactile)
Refer to iCub capabilities for a complete list of iCub capabilities.
Other Yarp modules required to effect this capability
camCalib controlGaze attentionSelection
Example instantiation
See:
iCub/scripts/startChicaEyeRight.bat iCub/scripts/startChicaSalienceRight.bat iCub/scripts/startChicaEgosphere.bat iCub/scripts/startChicaAttentionSelection.bat iCub/scripts/startChicaGazeControl.bat
Back to iCub YARP module specifications