VVV09 Control Gazers Group

From Wiki for iCub and Friends
Revision as of 11:02, 29 July 2009 by VVV09 (talk | contribs) (/controlGaze2/pos)
Jump to: navigation, search

Participants: (Please Join)

Alex Bernardino

Jonas Ruesch

Lorenzo Natale

David Vernon


ControlGaze2 is a module that controls in a coordinate way the head and the eyes of the iCub. It has been made for two main purposes:

  • Specify the gaze direction of the robot in (azimuth,elevation,vergence) angles rather that raw head/eye joints.
  • Provide a biological look to the dynamics of Head/Eye coordination.

The module allows changing the gaze direction by:

  • Specifying the absolute azimuth and elevation angles (in degrees).
  • Specifying the azimuth and elevation angles relative to the current gaze direction.
  • Specifying the pixel coordinates in the image where we want the robot to look at.
  • ... (TBC)

ControlGaze2 - Current State

Starting ControlGaze2

controlGaze2 <parameters>

To start control gaze you need to define the following parameters:



--configCamera iCubEyes.ini 

A file with camera parameters. To convert from pixel coordinates to azimuth and elevation angles we need the intrinsic parameters of the cameras. These are the parameters that result from camCalibConf. The file should be something like this:

projection pinhole
drawCenterCross 0
w  320
h  240
fx 224.34
fy 223.723
cx 178.58
cy 92.5934
k1 -0.381097
k2 0.153629
p1 0.0011246
p2 -0.00120395
projection pinhole
drawCenterCross 0
w  320
h  240
fx 214.953
fy 213.455
cx 166.773 
cy 125.505
k1 -0.358372
k2 0.116527
p1 -0.000603539
p2 -0.000591796 

If the images come from camCalib modules, the optical distortion parameters (k1, k2, p1, p2) should be put to 0.



--appPath c:/icub/app/myapp

The folder name where to find the previous file. This is subject to change given the new ResourceFinder specification.



--imageSize 160 120

If the image size is different from the specified in the calibration file, this allows rescaling the intrinsic parameters. If this is not specified, the module will use the image dimension specified in the calibration file.



--motorboard /icub/head

This specifies the prefix of the port names made available by the control board to the head device. If you use ICubInterface, this is usually /icub/head.

Other not so interesting parameters

pidON	0
pidGAIN	5.0
vorON	0
log	0
K 0.04 0 -0.08 -0.024 0 -0.056
FrameRate	20
ControlRate     50
limitResetTime 3.0
headSaccadeDelay 3.0

These could go to a initialization file (e.g. controlgaze2.ini)and invoked by:

--file controlgaze2.ini

This is probably going to change in future version due to the new ResourceFinder's methods.

Testing the module from the console

The module has a console interface where you can issue the following commands.

If you have access to the console window, you can just type the commands there. Otherwise open a command window and type:

yarp rpc /controlGaze2/conf

Then just type the required commands in the console.


   * sac abs <azi> <elev> : saccade in absolute angles (degrees)
   * set pos <azi> <elev> : same as before (obsolete but left here for historical reasons)
   * sac rel <azi> <elev> : saccade in head angles (degrees) relative to the current position
   * sac img <x> <y> : saccade in camera normalized relative coordinates (-1,1)
   * sac pix <x> <y> : saccade in image pixel coordinates 
   * pur abs <azispd> <elevspd> : pursuit in absolute angular velocities
   * pur rel <azispd> <elevspd> : pursuit in angular velocities relative to the current ones (acelerates, decelerates)
   * pur img <x> <y> : pursuit in camera normalized relative coordinates (-1,1)
   * pur pix <x> <y> : pursuit in image pixel relative coordinates 


   * verg abs <degrees>  : set the vergence angle 
   * verg rel <degrees>  : set the vergence angle with respect to the current one.
   * verg img <norm>     : set the vergence angle to compensate the existing disparity expressed in normalized coordinates.
   * verg pix <disp>     : set the vergence angle to compensate the existing disparity expressed in pixels.


   * rset : resets the state of the controller
   * get st : get time from last saccade
   * get stat : get controller status
   * get ref : get current gaze reference
   * get dh : get current head gaze
   * get der : get current right eye gaze
   * get del : get current left eye gaze 


   * get verg : get the current vergence angle (degrees)
   * get rv   : get the reference vergence angle (degrees)


   * set mst <float> : set minimum saccade time (a.k.a head saccade delay)
   * set lrt <float> : ser joint limit reset time
   * get mst         : get minimum saccade time (a.k.a head saccade delay)
   * get lrt         : get joint limit reset time
   * set verg <float>: sets the vergence controller gain
   * get verg        : gets the vergence controller gain

For instance, if you issue the command:

sac abs 10 10

The head-eye system will move to 10 degrees azimuth and 10 degrees elevation.

Sending Commands From Other Modules

You can issue the previous actuation commands from a module sending appropriate bottles to the /controlGaze2/conf port. For instance the following code will send a bottle instructing controlGaze2 to move to 10 degrees azimuth and 10 degrees elevation:

Bottle &bot = my_output_port.prepare();
bot.addVocab( Vocab::encode("sac") );
bot.addVocab( Vocab::encode("abs") );    

However, these commands use the RPC protocols and thus cannot be issued very fast (there is a delay due to the acknowledge/reply protocol). If you want to do send high frequency and low latency commands to the module, please use the streaming ports, described below.

Streaming Ports

Beyond /controlGaze2/conf, that receives the console commands, the other ports opened by controlGaze2 are of the streaming type, i.e. they do not provide replies or acknowledges.


This port accepts gaze position commands. The iCub head will perform a fast redirection of gaze (saccade), first moving the eyes quickly to the final heading and then moving the head, while the eyes counter-rotate to keep the image stabilized.

The module will ignore any position commands issued while performing the saccade. When the saccade finishes, other position commands can will be accepted.

To check if the module is accepting position commands, please see the documentation of port /status

Each command is a vector of 5 floats:

  • azimuth - the gaze value for the horizontal direction
  • elevation - the gaze value for the vertical direction
  • coord_sys - the type of coordinates/units related to the gaze values.
    • 'a' (cast to float)- the command is the absolute orientation (degrees).
    • 'r' (cast to float)- the command is the orientation relative to the current one (degrees).
    • 'p' (cast to float)- the command is given in normalized image coordinates (-1,1).
    • 'i' (cast to float)- the command is given in image pixel coordinates.
  • type_behav (optional)- just use 's', if you what to specify the following argument.
  • saccade_id (optional) - a increasing number specifying the saccade order. In some cases saccade commands can be out of order. This is sometimes the case if you stream saccade commands. The attention system uses this. Most probably you will not need it.



This port accepts gaze vergence commands. Each command is a vector of 1 float:

  • x - the horizontal image disparity in normalized image coordinates (-1, 1)






Outputs the state of the controller and the current azimuth and elevation gaze direction coordinates (degrees).

The port sends bottles with three elements:

  • status - an integer specifying the state of the controller:
    • 1 - starting a saccade (eye phase).
    • 2 - Continuing a saccade (neck phase).
    • 3 - Doing smooth pursuit.
    • 4 - Stuck in a limit.
    • 5 - Resting.
    • 6 - Starting smooth pursuit.
  • azimuth - a floating point number with the gaze direction azimuth (in degrees).
  • elevation - a floating point number with the gaze direction azimuth (in degrees).

The Future of ControlGaze2

ControlGaze3 ...

Call for Improvements

(Put here features that you would like to see in ControlGaze3)