# CrossPowerSpectrumVergence

Author: David Vernon

Determine the relative shift required to register one or more regions in two input images using the Fourier cross-power spectrum and use it to control the vergence of two cameras.

## Overview of module function

The cross-power spectrum of two images is defined as

```F(w_x, w_y) G*(w_x, w_y)
------------------------
|F(w_x, w_y) G(w_x, w_y)|
```

where `F(w_x, w_y)` and `G(w_x, w_y)` are the Fourier tranforms of images `f(x, y)` and `g(x, y)`, and `G*(w_x, w_y)` is the complex conjugate of `G(w_x, w_y)`

The positions of local maxima in the cross-power spectrum, specifically the offset of a detected maximum from the centre of the CPS image, indicates the relative image shift required to register regions in the image. This can be used to control the vergence of the two eyes.

Typically, there will be several regions in the image with different disparities and hence different vergence angles, each with its own corresponding maximum, we need a strategy to choose one of the maxima to control the fixation/vergence

The possibilities are:

• choose the largest maximum; this is probably going to correspond to the object that occupies the largest amount of the field of view (or largest energy in the image)
• choose the maximum that is closest to the centre; the corresponds to the object that is closest to the current fixation distance
• choose the maximum that is furthest to the LEFT of the cross-power spectrum; this corresponds to the object that is closest to the cameras

We use the third option at the moment.

## Usage

### Module arguments

```--file <filename> // see #Configuration files
```

Port name control:

``` --name NAME:       changes prefix of port names to NAME
```

Default port names are:

```/left_camera:i  (for incoming images from left camera)
/right_camera:i (for incoming images from rightcamera)
```

### Ports created

Ports that are instantiated by the module and are then available for other modules to use (using yarp connect)

Under normal use, the following ports are created.

#### Input ports

``` /left_camera:i        images from left camera: ImageOf<PixelRgb>
/right_camera:i       images from left camera: ImageOf<PixelRgb>
```

#### Output ports

```
/left_image:o                     filtered images from left and right camera:
/right_image:o                    converted to square aspect ratio to facilitate FFT
and apodized with a Gaussian functionto facilitate centre weighting

/cross-power_spectrum:o           the raw cross-power spectrum of the two filtered images

/filtered_cross-power_spectrum:o  a filtered version with non-maxima suppression to enhance
the maxima detection process.
Detected maxima are denoted by cross-hairs.

/vergence_disparity:o             a stream of Vectors comprising three double entities:
the dx (horizontal) and dy (vertical downwards) disparity values
together with a flag value to denote the type of image use (default value of 0)
The disparity values are expressed in normalized values -1 <= dx, dy <= +1
```

### Ports accessed

Ports that are assumed to exist prior to instantiation of the module (i.e. some other module must create them)

There needs to be two camera inputs available.

```     /icub/cam/left
/icub/cam/right
```

Eye control is actually performed by the controlGaze module.

Thus, crossPowerSpectrumVergence is connected to controlGaze via the port

```     vergence_disparity:o
```

None

None

### Configuration files

Module arguments can read from a configuration file by specifying `--file <filename>.ini`.

The following are the contents of the `crossPowerSpectrumVergence.ini` file.

```std_dev                         20
threshold                       10
number_of_maxima                 2
```

These are also the default values hard-coded into the module.

• `std_dev` is the standard deviation of Gaussian used for centre weighting, specified in % of image width.
• `threshold` is the threshold for detection of maxima, specified as an integer % of the global maximum.
• `filter_radius` is the radius of the filter used to amplify local maxima, specified in pixels
• `number_of_maxima` is the maximum number of maxima to be detected.
• `non_maxima_suppression_radius` maxima within this radius of a selected maximum are suppressed, specified in pixels

### User interface mechanism

Run-time modification of module parameters

The module is intended to run autonomously.

## Example instantiation

Run & port connection commands

yet to be included