- The correct title of this article is crossPowerSpectrumVergence. The initial letter is shown capitalized due to technical restrictions.
Author: David Vernon
Determine the relative shift required to register one or more regions in two input images using the Fourier cross-power spectrum and use it to control the vergence of two cameras.
Back to iCub YARP module specifications
Overview of module function
The cross-power spectrum of two images is defined as
F(w_x, w_y) G*(w_x, w_y) ------------------------ |F(w_x, w_y) G(w_x, w_y)|
F(w_x, w_y) and
G(w_x, w_y) are the Fourier tranforms
f(x, y) and
g(x, y), and
G*(w_x, w_y) is the
complex conjugate of
The positions of local maxima in the cross-power spectrum, specifically the offset of a detected maximum from the centre of the CPS image, indicates the relative image shift required to register regions in the image. This can be used to control the vergence of the two eyes.
Typically, there will be several regions in the image with different disparities and hence different vergence angles, each with its own corresponding maximum, we need a strategy to choose one of the maxima to control the fixation/vergence
The possibilities are:
- choose the largest maximum; this is probably going to correspond to the object that occupies the largest amount of the field of view (or largest energy in the image)
- choose the maximum that is closest to the centre; the corresponds to the object that is closest to the current fixation distance
- choose the maximum that is furthest to the LEFT of the cross-power spectrum; this corresponds to the object that is closest to the cameras
We use the third option at the moment.
--file <filename> // see #Configuration files
Port name control:
--name NAME: changes prefix of port names to NAME
Default port names are:
/left_camera:i (for incoming images from left camera) /right_camera:i (for incoming images from rightcamera)
Ports that are instantiated by the module and are then available for other modules to use (using yarp connect)
Under normal use, the following ports are created.
/left_camera:i images from left camera: ImageOf<PixelRgb> /right_camera:i images from left camera: ImageOf<PixelRgb>
/left_image:o filtered images from left and right camera: /right_image:o converted to square aspect ratio to facilitate FFT and apodized with a Gaussian functionto facilitate centre weighting /cross-power_spectrum:o the raw cross-power spectrum of the two filtered images /filtered_cross-power_spectrum:o a filtered version with non-maxima suppression to enhance the maxima detection process. Detected maxima are denoted by cross-hairs. /vergence_disparity:o a stream of Vectors comprising three double entities: the dx (horizontal) and dy (vertical downwards) disparity values together with a flag value to denote the type of image use (default value of 0) The disparity values are expressed in normalized values -1 <= dx, dy <= +1
Ports that are assumed to exist prior to instantiation of the module (i.e. some other module must create them)
There needs to be two camera inputs available.
Eye control is actually performed by the controlGaze module.
Thus, crossPowerSpectrumVergence is connected to controlGaze via the port
Input data files
Output data files
Module arguments can read from a configuration file by specifying
The following are the contents of the
std_dev 20 threshold 10 filter_radius 2 number_of_maxima 2 non_maxima_suppression_radius 10
These are also the default values hard-coded into the module.
std_devis the standard deviation of Gaussian used for centre weighting, specified in % of image width.
thresholdis the threshold for detection of maxima, specified as an integer % of the global maximum.
filter_radiusis the radius of the filter used to amplify local maxima, specified in pixels
number_of_maximais the maximum number of maxima to be detected.
non_maxima_suppression_radiusmaxima within this radius of a selected maximum are suppressed, specified in pixels
User interface mechanism
Run-time modification of module parameters
The module is intended to run autonomously.
Run & port connection commands
yet to be included
Back to iCub YARP module specifications