From Wiki for iCub and Friends
Revision as of 14:23, 21 November 2014 by Gsaponaro (talk | contribs) (If you have compiled from source: OpenNI2 64-bit and 32-bit)
Jump to: navigation, search

The OpenNI2 device driver for YARP works with PrimeSense compatible RGB-D sensors, including Asus Xtion Pro. Tested on Debian Jessie (testing), Ubuntu 12.04+ and Mac OS X 10.8.4+.

Debian Wheezy (stable) is not supported by OpenNI2, due to glibc version being < 2.14.

Note: Microsoft Kinect sensors are not supported by this driver. However, they are supported by (1), based on OpenNI version 1 or Microsoft Kinect SDK, and (2) the old Kinect YARP driver, also based on OpenNI 1.



For Debian and Ubuntu, install the following dependencies:

sudo apt-get install g++ python libusb-1.0-0-dev libudev-dev freeglut3-dev doxygen graphviz

Also, you must have installed a Java SDK.

For MacOS X, the above packages are either preinstalled or available through Homebrew.

Next, install OpenNI2 (from zip packages or from GitHub source - see and NiTE2.

Environment variables

Next, you need to set four new environment variables in your system, typically in your .bashrc:


If you have downloaded the packages

Source the environment files (obtained from the zip packages instructions) by copying their content permanently into your .bashrc. From your OpenNI2 directory:

cat OpenNIDevEnvironment >> ~/.bashrc

And from your NiTE2 directory:

cat NiTEDevEnvironment >> ~/.bashrc

If you have compiled from source

Manually set the following in your .bashrc:

OPENNI2_REDIST=/full/path/to/OpenNI2/x64-Release or /full/path/to/OpenNI2/x86-Release
// NITE2 variables are set from binary packages, see previous step

Enabling OpenNI2 YARP driver

In the CMake configuration for YARP, set the following:

ENABLE_yarpmod_OpenNI2DeviceServer ON
ENABLE_yarpmod_OpenNI2DeviceClient ON

We recommend that you also turn on RPATH (not strictly necessary, but useful to avoid cumbersome file copying of driver files - see next section). In CMake, press t to toggle advanced options, then set:


If your environment variables were defined correctly, the following settings will be automatically taken care of by CMake:

NITE2_INCLUDE_LOCAL: full path to NiTE-2.x.x/Include directory
NITE2_LIBRARY: full path to
OPENNI2_INCLUDE_LOCAL: full path to OpenNI-2.x.x-arch/Include directory (arch is x64 or x86)
OPENNI2_LIBRARY: full path to

Finalizing the installation

These final steps are necessary to not incur in runtime errors like Couldn't create user tracker. If you use RPATH (recommended option), you need to copy and configure one file. If you do not use RPATH, you need to copy two directories. Details follow.

If you have set RPATH in the previous step

In this case, the OpenNI2 YARP driver must always be called from a directory that contains a copy (or symbolic link) of the NiTE.ini file from the stock NiTE2 distribution, properly configured. To prepare such a setup, do something along these lines:

mkdir ~/yarp_openni2_root
cd ~/yarp_openni2_root
cp $NITE2_REDIST64/NiTE.ini .
// or if you have the 32-bit version:
// cp $NITE2_REDIST/NiTE.ini .

Edit NiTE.ini, setting DataDir to be the full path to $NITE2_REDIST64/NiTE2 (or $NITE2_REDIST/NiTE2).

Now you're ready to call the yarpdev command from the yarp_openni2_root directory.

If you have not set RPATH in the previous step

In this case, the OpenNI2 YARP driver must always be called from a directory that contains a copy (or symbolic link) of the stock OpenNI2 and NiTE2 'Drivers' directories. To prepare such a setup, do something along these lines:

mkdir ~/yarp_openni2_root
cd ~/yarp_openni2_root
cp -R $NITE2_REDIST64/NiTE2 .
// or if you have the 32-bit version:
// cp -R $NITE2_REDIST/NiTE2 .

and call the yarpdev command from the yarp_openni2_root directory.


To print a list of all available options (help):

yarpdev --device OpenNI2DeviceServer --verbose

General options:

option description default
--name Set port prefix. /OpenNI2
--printVideoModes Print supported video modes.
--depthVideoMode Set depth camera video mode. 0 (i.e., 320x240 @ 30fps)
--colorVideoMode Set color (RGB) camera video mode. 0 (i.e., 320x240 @ 30fps)
--noCameras Disable camera output, while keeping user skeleton enabled. Useful to reduce CPU load. off (i.e., cameras on)
--noMirror Disable mirroring. Mirroring causes the video stream to appear as if seen in a mirror, i.e., the image is transformed by reflecting all pixels across the vertical axis. off (i.e., mirroring on)
--noUserTracking Disable skeleton user tracking. Useful to reduce CPU load. off (i.e., user tracking on)
--minConfidence x (double) Set minimum confidence required by skeleton user tracking. 0.6
--syncFrames Keep color and depth frames always in temporal sync. off
--imageRegistration Align color and depth video streams. This computation is performed on the sensor hardware. off

Other options related to recording and playback:

option description default
--record filename.oni Record (save) to the specified oni file.
--playback filename.oni Playback the specified oni file.
--loop Force the playback to loop. off (i.e., playback stops after one run)

Note: in playback, options like --depthVideoMode and --colorVideoMode are ignored.

Output and data format

The driver opens these ports:

  • [portPrefix]/userSkeleton:o - userSkeleton detection port (only opened if user detection is on)
  • [portPrefix]/depthFrame:o - depth frame port
  • [portPrefix]/imageFrame:o - rgb camera frame port

imageFrame:o and depthFrame:o stream YARP images (PixelRGB and PixelMono16, respectively), and if connected to a yarpview, the video is available.

During tracking, userTracker:o contains YARP Bottles in the following format:

([USER] userID) [POS] (x y z) positionConfidence [ORI] (w x y z) orientationConfidence ...


  • the part between "[POS]" and "orientationConfidence" is repeated as many times as the number of skeleton joints (15: HEAD NECK LEFT_SHOULDER RIGHT_SHOULDER LEFT_ELBOW RIGHT_ELBOW LEFT_HAND RIGHT_HAND TORSO LEFT_HIP RIGHT_HIP LEFT_KNEE RIGHT_KNEE LEFT_FOOT RIGHT_FOOT)
  • the position is expressed in 3D Cartesian coordinates
  • the orientation is expressed in four-dimensional quaternions

Using the driver from an application

Refer to