EgoSphere

From Wiki for iCub and Friends
Jump to: navigation, search
The correct title of this article is egoSphere. The initial letter is shown capitalized due to technical restrictions.


Author: Jonas Ruesch, Alexandre Bernardino

Integrates visual and acoustic sensory data to provide a combined visuo-acoustic saliency map.


Note that this saliency map is defined in spherical frame of reference with two degrees of freedom: azimuth and elevation, with the origin at the base of the neck. These degrees of freedom are not directly compatible with the iCub neck joints (neck_roll, neck_pitch, neck_yaw). However, the controlGaze module, with the help of the kinematics library, has a mode the convert from azimuth/elevation coordinates to actual neck joint angles. Zero position is 'looking straight ahead' (zero elevation, zero azimuth).


Webpage for this module: http://eris.liralab.it/iCub/dox/html/group__icub__egosphere.html


Back to iCub YARP module specifications


Egosphere.jpg


Compiler & Linker Dependencies

Include files

  #include <iCub/EgoSphereModule.h>
  #include <iCubSphereProjector.h>
  #include <iCub/RemoteEgoSphere.h>
  #include <iCub/EgoSphereInterfaces.h>


Libraries

  YARP
  OpenCV
  GSL        (GNU Scientific Library)
  spmap      (iCub library)
  kinematics (iCub library)

Run-time Dependencies

Module arguments

--file <filename> // see #Input data files

If desired, command line arguments can be moved to a configuration file by specifying --file NAME.ini. This is useful for complicated configurations. Here is an example ini file:

  name /chica/egosphere
  group EGO_SPHERE
  groupCamCalib CAMERA_CALIBRATION_RIGHT
  nameControlboard /controlboard

  [EGO_SPHERE]

  resx    320
  resy    240
  weightVision 1.0
  weightSound 0.0
  activateFloatMosaicVisualization 1
  drawSaliencePeak 1
  visualUpdate 1
  refreshTime 0.05
  salienceDecayRate 0.995
  thresholdSalience 20.0

  [CAMERA_CALIBRATION_RIGHT]
  … // see camCalib module


Ports accessed

Ports that are assumed to exist prior to instantiation of the module (i.e. some other module must create them)


Ports created

Ports that are instantiated by the module and are then available for other modules to use (using yarp connect)

Input ports

  /egosphere/map             Visual salience input map (floating point image) in camera coordinates. 
                             Usually will be streamed from salience module to egosphere.

  /egosphere/rgb             RGB image to visualize a spherical mosaic (monitoring purpose only); camera coordinates

  /egosphere/sound           Vector of detected sound sources:
                             Input from soundLocalization module 
                             (planned format for a sound source in head coordinate-system:
                             [elevation, azimuth, covarianceElevation, covarianceAzimuth, Intensity]

  /egosphere/controlboard    Connects on startup to a server control board to read encoder values  
                             in order to compute gaze direction 
                             (server control board port specified by –nameControlboard option) 

Output ports

  /egosphere/ego/map         The output saliency map. 
                             A spherical floating point image x->azimuth, y->elevation

  /egosphere/ego/view        RGB visualization of the floating point saliency map (active only if configured)

  /egosphere/ego/rgb         A rendering of the visual data mapped onto the spherical frame of reference;
                             this will comprise a mosaic of camera views.               
                             It is created only if input provided to /egoshpere/rgb

                             All these representations are defined in a spherical frame of reference, 
                             in units of degrees, ranging left to right from -180 deg. to +180 deg. 
                             and bottom to top from -90 deg. to +90 deg.

Inline dotgraph 5.png

Input data files

Needs access to a camera calibration option group (e.g. available from camcalibconf in chica_eyes.ini)

To explain: sometimes it is convenient to have different modules accessing the same configuration file. To facilitate this, the configuration file can contain different configuration groups for different modules (groups are sections started by a tag like [GROUP_NAME] in an *.ini file). For example, the configuration group for the intrinsic camera parameters needs to be accessed by different modules, it would be poor practice to have copies of the same configuration parameters in different files. The egoSphere needs a configuration group [EGO_SPHERE] but additionally also the [CAMERA_CALIBRATION] group.

Output data files

None.

Configuration files

  iCub/conf/chica_eyes.ini

User interface mechanism

Run-time modification of module parameters

User interface is effected currently through module terminal, e.g. by typing

  shell> set vu 0     // switch visual update off
  shell> set ts 20.0  // set salience threshold to 20
  shell> set sd 0.99  // set salience decay (memory decay)
  shell> set wv 0.7   // set vision map weight
  shell> set ws 0.3   // set sound map weight

All above properties can be returned by ‘get xy’

Possible graphical user interfaces can use the c++ class RemoteEgoSphere in order to access the egoSphere module.

OS on which the module was developed

Windows


OS on which the module was tested

Windows (& compilation on Linux)


Operating system dependencies

None known.


Example instantiation of the module

Run & port connection commands

REM Copyright (C) 2007 Jonas Ruesch
REM CopyPolicy: Released under the terms of the GNU GPL v2.0.

REM Simple example instantiation for the camCalib module.
REM You need to adapt values between <> according to your environment in order to make this work.
REM Depending on your configuration you may not want to use all of the connections/viewers below.
REM The module configuration is loaded from a file specified by the --file option. To create this file
REM have a look at the default terminal output of the module. A module reports it's option keys 
REM and default values on startup. Some modules require an additional option --group which specifies
REM the name of the configuration group within the configuration file (specify without []).
REM We assume you compiled in release mode.

:: Start module
start <ICUB_DIR>\bin\release\egosphere.exe --name /iCub/egoSphere --file <x:/path/to/config.ini> --group <GROUP_NAME_IN_CONFIG_FILE> --groupCamCalib 

<CONFIG_GROUP_CAMCALIB> --nameControlboard <NAME_PORT_SERVER_CONTROLBOARD>
:: Start viewers 
start yarpview --RefreshTime 29 --name /view/egoSphere/ego/rgb --out /view/egoSphere/ego/rgb/click
start yarpview --RefreshTime 29 --name /view/egoSphere/ego/view

:: Connect sources 
yarp wait /iCub/egoSphere/rgb
yarp connect </camCalib/image> /iCub/egoSphere/rgb
yarp wait /iCub/egoSphere/map
yarp connect </salience/map> /iCub/egoSphere/map
yarp wait /iCub/egoSphere/sound
yarp connect </soundLocalization/o:vct> /iCub/egoSphere/sound
:: Connect viewers
yarp wait /view/egoSphere/ego/rgb
yarp connect /iCub/egoSphere/ego/rgb /view/egoSphere/ego/rgb
yarp wait /view/egoSphere/ego/view
yarp connect /iCub/egoSphere/ego/view /view/egoSphere/ego/view
:: Connect viewer click ports
yarp wait /iCub/egoSphere/ego/click
yarp connect /view/egoSphere/ego/rgb/click /iCub/egoSphere/ego/click

See also iCub/scripts/startChicaEgosphere.bat


iCub Capabilities

iCub capability code

C3   Learn to reach towards a fixation point
C6   Construct sensorimotor maps & cross-modal maps
C8   Exploratory, curiousity-driven, action
C10  Crawl and navigate based on local landmarks and ego-centric representations
C11  Saccadic direction of gaze towards salient events (visual, auditory, tactile)
C12  Focus attention and direct gaze on human faces (features, movements, sounds)

Refer to iCub capabilities for a complete list of iCub capabilities.

Other Yarp modules required to effect this capability

camCalib
controlGaze
salience
attentionSelection


Example instantiation

See:

iCub/scripts/startChicaEyeRight.bat
iCub/scripts/startChicaSalienceRight.bat
iCub/scripts/startChicaEgosphere.bat
iCub/scripts/startChicaAttentionSelection.bat
iCub/scripts/startChicaGazeControl.bat


Back to iCub YARP module specifications