CrossPowerSpectrumVergence

From Wiki for iCub and Friends
Jump to navigation Jump to search
The correct title of this article is crossPowerSpectrumVergence. The initial letter is shown capitalized due to technical restrictions.

Author: David Vernon

Determine the relative shift required to register one or more regions in two input images using the Fourier cross-power spectrum and use it to control the vergence of two cameras.


Back to iCub YARP module specifications


Overview of module function

The cross-power spectrum of two images is defined as

F(w_x, w_y) G*(w_x, w_y)
------------------------
|F(w_x, w_y) G(w_x, w_y)|


where F(w_x, w_y) and G(w_x, w_y) are the Fourier tranforms of images f(x, y) and g(x, y), and G*(w_x, w_y) is the complex conjugate of G(w_x, w_y)

The positions of local maxima in the cross-power spectrum, specifically the offset of a detected maximum from the centre of the CPS image, indicates the relative image shift required to register regions in the image. This can be used to control the vergence of the two eyes.

Typically, there will be several regions in the image with different disparities and hence different vergence angles, each with its own corresponding maximum, we need a strategy to choose one of the maxima to control the fixation/vergence

The possibilities are:

  • choose the largest maximum; this is probably going to correspond to the object that occupies the largest amount of the field of view (or largest energy in the image)
  • choose the maximum that is closest to the centre; the corresponds to the object that is closest to the current fixation distance
  • choose the maximum that is furthest to the LEFT of the cross-power spectrum; this corresponds to the object that is closest to the cameras

We use the third option at the moment.


Usage

Module arguments

--file <filename> // see #Configuration files

Port name control:

 --name NAME:       changes prefix of port names to NAME

Default port names are:

/left_camera:i  (for incoming images from left camera)
/right_camera:i (for incoming images from rightcamera)


Ports created

Ports that are instantiated by the module and are then available for other modules to use (using yarp connect)

Under normal use, the following ports are created.

Input ports

 /left_camera:i        images from left camera: ImageOf<PixelRgb>
 /right_camera:i       images from left camera: ImageOf<PixelRgb>


Output ports


  /left_image:o                     filtered images from left and right camera:
  /right_image:o                    converted to square aspect ratio to facilitate FFT
                                    and apodized with a Gaussian functionto facilitate centre weighting 

  /cross-power_spectrum:o           the raw cross-power spectrum of the two filtered images

  /filtered_cross-power_spectrum:o  a filtered version with non-maxima suppression to enhance
                                    the maxima detection process.  
                                    Detected maxima are denoted by cross-hairs.

  /vergence_disparity:o             a stream of Vectors comprising three double entities:
                                    the dx (horizontal) and dy (vertical downwards) disparity values
                                    together with a flag value to denote the type of image use (default value of 0)
                                    The disparity values are expressed in normalized values -1 <= dx, dy <= +1                   


Ports accessed

Ports that are assumed to exist prior to instantiation of the module (i.e. some other module must create them)

There needs to be two camera inputs available.

     /icub/cam/left
     /icub/cam/right

Eye control is actually performed by the controlGaze module.

Thus, crossPowerSpectrumVergence is connected to controlGaze via the port

     vergence_disparity:o 


Input data files

None

Output data files

None

Configuration files

Module arguments can read from a configuration file by specifying --file <filename>.ini.

The following are the contents of the crossPowerSpectrumVergence.ini file.

std_dev                         20
threshold                       10
filter_radius                    2
number_of_maxima                 2
non_maxima_suppression_radius   10

These are also the default values hard-coded into the module.

  • std_dev is the standard deviation of Gaussian used for centre weighting, specified in % of image width.
  • threshold is the threshold for detection of maxima, specified as an integer % of the global maximum.
  • filter_radius is the radius of the filter used to amplify local maxima, specified in pixels
  • number_of_maxima is the maximum number of maxima to be detected.
  • non_maxima_suppression_radius maxima within this radius of a selected maximum are suppressed, specified in pixels


User interface mechanism

Run-time modification of module parameters

The module is intended to run autonomously.


Example instantiation

Run & port connection commands

yet to be included


Back to iCub YARP module specifications