From Wiki for iCub and Friends
Jump to navigation Jump to search
The correct title of this article is tracker. The initial letter is shown capitalized due to technical restrictions.

Author: Paul Fitzpatrick

Visual tracker. Tracks a single visual target using input from a single camera. If mapping from visual coordinates to motor commands is not known, an adaptation mode can be selected, which will make small exploratory movements.

tracker implements tracking in a low-level sense, not any of the many possible high-level senses. Specifically: if the robot is looking at something in this instant, "tracker" tries to figure out how to keep the robot looking at the same thing in the next instant, despite movements of the object and/or head.

tracker is meant to work with other modules. It does not decide what to look at, when to look at it, and when to look away. That's the job of other modules (e.g. salience + attentionSelection + ...).

By implementing low-level tracking in a distinct module, the control-loop can be kept tight, and the rest of the system can react to changes in the environment at a slower or more variable rate without making the robot's gaze seem jerky and unpredictable. Basically, we transform the default low-level behavior of the robot's visual system (in the absence of commands from higher-up) from being "look in the same direction you are currently looking" to being "keep looking at what you're looking". This is almost always what you want, and corresponds well to the default reaction in human vision too.

Finally, tracker does not need to directly control the eyes, it is configured to pass on its suggestions to another module. Basically, it provides a stream that says "if you want to keep the eyes looking at the same object, move them like this". So it could be an input to something like controlGaze or simpleHeadcontrol or whatever actually gets to talk to the physical hardware. However, it shouldn't be too far removed from the real controller or latencies will cause ugly behavior. This allows for switching to other modes of operation (e.g. ballistic saccades, if desired) without contaminating the tracker with new functionality, thereby keeping each module cohesive.

In summary, Tracker is cross-correlation based image patch matching module that produces image frame-of-reference positional offsets that can be mapped by another module to joint control offsets or control point or whatever. These offsets can (optionally) be adaptively weighted if they don't inially match well with the controller constants/gains. The initial seed position from which the initial patch to match (track) is provided by, e.g., salience+attentionSelection, but probably not egosphere (because it's frame of reference is neck based in units of degrees, rather than image based in units of pixels). The output of tracker can be used selectively by any intermediary module to control the gaze (eyes, head, ..), or ignored should a higher priority control strategy kick in, typically re-initializing tracker with a new position/patch to track.

Back to iCub YARP module specifications

Compiler & Linker Dependencies

Include files

YARP    // miscellaneous
STL     // for <deque>



Run-time Dependencies

Module arguments

--file <filename> // see #Input data files

Port name control:

 --name NAME:       changes prefix of port names to NAME
 --image_name NAME: changes just the image port name
 --pos_name NAME:   changes just the position port name
 --enc_name NAME:   changes just the encoder port name

Default port names are:

/tracker/img (for incoming and outgoing images)
/tracker/pos (for streaming position vector output)

Input format control:

 --vvv: for "summer school" input format

Output format control:

 --rel: for "[set] [rels] ($x $y)" input format

Adaptivity control:

 --monitor: turns on adaptation behavior, where tracker monitors system response to its commands 
            and modifies commands to get desired response.

Ports created

Ports that are instantiated by the module and are then available for other modules to use (using yarp connect)

Under normal use, the following ports are created. The port names are just examples (controlled by options):

Input ports

    /icub/tracker/img: This port expects to receive a stream of 
                       wide-field-of-view images from a camera.

    /icub/tracker/pos: This port expects to receive occasional commands 
                       resetting the tracker or changing the area to be tracked.

    /icub/tracker/enc: This (optional) port expects to receive a stream of encoder readings 
                       from the motors.  This is required if adaptation is enabled.

Output ports

(overlap with input ports)

    /icub/tracker/img: This port generates an annotated copy of the stream of images received, 
                       showing the location of the tracked area.  
                       This is intended for human monitoring purposes only.

    /icub/tracker/pos: This port generates a stream of locations, corresponding to the 
                       currently tracked area's position.  Locations can be expressed as 
                       simple coordinates or mapped onto motor commands.

Ports accessed

Ports that are assumed to exist prior to instantiation of the module (i.e. some other module must create them)

There needs to be a wide-field-of-view-camera input available. On iCub, the tradition seems to be to call this:


There needs to be something to control the head movement; ideally there is also a way to measure how much the head actually moved (as opposed to how it was commanded).

On iCub, control and state for the head [1] is available at:


Typically, the tracker is not connected directly to control the head; there is usually an intermediary module called "simple_headcontrol" [2]. This is usually

configured to create a port called:


That port, for historical reasons, accepts messages in "summer school format" (generated by --vvv flag to tracker).

So, to summarize, the external ports and typical connections are:

     /icub/cam/left  -->  /icub/tracker/img
     /icub/head/state:o  -->  /icub/tracker/enc
     /icub/tracker/pos  -->  /icub/target

The tracker will work without an external position command input, and is frequently used this way for demos. If an attention system is available, it can be connected

as an input to:




Input data files


Output data files


Configuration files

If desired, command line arguments can be moved to a configuration file by specifying --file NAME.ini.

User interface mechanism

Run-time modification of module parameters

The module is intended to run autonomously. For testing, new targets to track can be sent to the module by hooking up the click output of a yarpview process to the

modules' motor input.

OS on which the module was developed


OS on which the module was tested

Linux, Windows

Operating system dependencies

None known.

Example instantiation of the module

Run & port connection commands

(run commands in separate terminals when necessary)

(syntax needs to be a little different on Windows vs Linux)

(ICUB_ROOT = location of iCub code, YARP_ROOT = location of yarp)

 cd $ICUB_ROOT/src/tracker
 ./tracker --name /icub/tracker --rel
 yarpview --name /viewer --out /clicker
 yarp connect /icub/cam/left /icub/tracker/img
 yarp connect /icub/tracker/img /viewer
 yarp connect /clicker /icub/tracker/pos

New standard avaiable

now tracker has a new standard for /tracker/pos output port it is possible to use it using the --pmONE option, like:

./tracker --name /icub/tracker --pmONE

iCub Capabilities

iCub capability code

C25  Gaze control: smooth pursuit (with prediction); possibly tuned by learning
(but no prediction; some crude adaptation)

Other Yarp modules required to effect this capability

yarpdev (for image source, motor output)

Example instantiation

For testing, it is easiest to instantiate the module and connect it to the "fakebot" (

cd $YARP_ROOT/examples/tutorial
./run_fakebot --file admin.ini
yarp connect /fakebot/camera /icub/tracker/img
yarp connect /icub/tracker/pos /fakebot/rpc:i


The tracker for fakebot boat example, in 8 steps:

0-(remember to run "yarp server" somewhere)

1-launch fakebot: ./run_fakebot --file admin.ini

2-launch yarpview: yarpview --name /viewer --out /clicker

3-launch the tracker: ./tracker --name /tracker --rel

4-connect yarpview and the camera of fakebot: yarp connect /fakebot/camera /viewer

5-connect tracker and fakebot's camera: yarp connect /fakebot/camera /tracker/img

6-connect yarpview and the tracker: yarp connect /tracker/img /viewer

7-connect the clicker and the tracker: yarp connect /clicker /tracker/pos

8-connect fakebot motor input and tracker position output: yarp connect /tracker/pos /fakebot/motor/rpc:i


The tracker for iCub's head, in 9 steps:

(I suppose you are connected with the head's dragonfly camera :o) )

0-(remember to run "yarp server" somewhere)

1-launch iCubInterface: ./iCubInterface --file icubHeadOnly.ini

2-launch yarpview: yarpview --name /viewer --out /clicker

3-launch the tracker: ./tracker --name /tracker --vvv

4-launch the controller: ./simpleHeadControl --robot icub --part head

5-launch the grabber: yarpdev --device dragonfly --d 0 --width 320 --height 240 --white_balance 20 50 --gain 600 --name /icub/cam/left --port_number 1 --shutter 0.023

6-connect yarpview and iCub's camera: yarp connect /icub/cam/left /tracker/img

7-connect yarpview and tracker: yarp connect /tracker/img /viewer

8-connect clicker and tracker: yarp connect /clicker /tracker/pos

9-connect motor input and tracker position output: yarp connect /tracker/pos /icub/target

some information about what you are trying to do:

Remember that: if you are working inside a net where other people can use the same hardware you are going to use, probably, is better to call the ports with your USERNAME inside the nameport,

for example: instead of "yarpview --name /viewer --out /clicker" write "yarpview --name /MATTIA/viewer --out /MATTIA/clicker" r

0-usually you have not to run "yarp server" because usually, each lab which uses yarp, has a server running. Yarp is able to check if someone has launched "yarp server" before you. To be sure about it you can write "yarp where" on your shell to check it.

1-ICubInterface, in this case, is going to open 3 ports:

* /icub/head/rpc:i
* /icub/head/command:i
* /icub/head/state:o

3-tracker, in this case, is going to open:

* /tracker/img
* /tracker/pos
* /tracker/enc

4-simpleHeadControl, in this case, is going to open 5 ports:

* /icub/head/control/rpc:o
* /icub/head/control/command:o
* /icub/head/control/state:i
* /icub/target
* /icub/inertial_data

5-some options have been added to yarpdev, for example, you can select the firewirecamera card you are going to use, in case your machine has more than one. the command is "port_name NumberOfThePort". Usually number 0 is used if you have just 1 firewire card and number 1 can be useful if you are using 2 cards.

Remember that:

  • yarpdev has to be launched from the machine where the dragonfly card is connected

p.s. you can use the option "--shutter" to manage the brightness of your image, or you can run the program frameGrabberGui which helps you to manage Brightness, Gain, Shutter, Red and Blue online.

Back to iCub YARP module specifications