Kinect

Background
Fetching Data
Method 1: OpenNI Middleware
Method 2: ROS

Background

Fetching Data

There are two ways to retrieve data from the kinect.

  1. OpenNI Middleware
  2. ROS

There are advantages and disadvantages to both methods. For the sake of MEAM620 - you need to be able to retrieve the IR image for Phase1 of Project2 so you should use the OpenNI Middleware for that. The openni_kinect stack has not yet made IR image available.

Method 1: OpenNI Interface

Download Andrew's Slides (ppt)
Download Andrew's Slides (pdf)

Installing the Driver

There are 3 parts to the installation.

  1. OpenNI Middleware
  2. Kinect Driver
  3. NITE Software
    (We may not get up to the point to where we will need it this semester)

Below are links to download the *unstable* version of each piece of software.

OpenNI Middleware

Scroll down and download the software for your platform:
http://www.openni.org/downloadfiles/openni-binaries/20-latest-unstable

For Platforms that aren't listed there:
OpenNI-OpenNI-4c9ff97.zip

Kinect Driver

For all platforms:
avin2-SensorKinect.zip

NITE

Scroll down and download the software for your platform:
http://www.openni.org/downloadfiles/openni-compliant-middleware-binaries/33-latest-unstable

Using the API

Documentation: [OpenNi Path]/Documentation/html/main.html
Examples: [OpenNi Path]/Samples
Code to Save Data to File: Attach:KinectCollectData.tar.gz

  • Extract folder and type "make" to compile
  • Configured for collecting high res IR data
  • Run CollectData
  • It will ask for a file name and number of frames to grab
  • After it finishes, load data into matlab using provided functions. See sampleRead.m for an example.
API Basics
  1. Initialization
    • Create XML Configuration file: see [OpenNi Path]/Samples/Config/SamplesConfig.xml
    • Load initialization file: XnStatus xn::Context::InitFromXmlFile
    • Find Depth,Image, or IR Generators: XnStatus xn::Context::FindExistingNode
  2. Wait for New Data
    • XnStatus xn::Context::WaitAndUpdateAll
    • XnStatus xn::Context::WaitAnyUpdateAll
    • XnStatus xn::Context::WaitNoneUpdateAll
    • XnStatus xn::Context::WaitOneUpdateAll
  3. Retrieve Data
    • XnRGB24Pixel* xn::ImageGenerator::GetRGB24ImageMap
    • XnIRPixel* xn::IRGenerator::GetIRMap
    • XnDepthPixel* xn::DepthGenerator::GetDepthMap

Method 2: ROS openni_kinect Stack

The openni_kinect stack provides an interface to stream data from the kinect and publish the image data to various topics.

The stack is only supported under Diamondback so you need to install Diamondback first and then the stack.

If you are planning on using C++/Python for your project, then you can move on to learning how to use your kinect.

If you are planning on using Matlab, you need to follow the directions below. The directions explain how you can fix your IPC-Bridge that you installed for Project1 so that it works with Project2. Please read them carefully because while they are describe very easy steps you must take they may be confusing. Email Ben with any corrections.

NOTE: You must follow this procedure if you've installed the ipc-bridge on your machine for Project 1 (Before 3/28/11). If you are installing it afterwards, then this problem doesn't exist.

A problem with the IPC-Bridge

A bug (not exactly a bug, more of an incompatibility) has been found in IPC that causes the IPC-Bridge to crash when bringing up "sensor_msgs_Image_publisher" nodes. The problem arises when you want to subscribe to a ROS topic that publishes sensor_msgs/Image from within Matlab. The problem is only with the sensor_msgs/Image message definition which is used for passing around images (ex. getting images from a camera). This problem was fixed in the new version of the ipc-bridge linked to on the ipc-bridge page.

Prerequisites

  • Diamondback is installed.
    • You installed Diamondback already. If not, follow this tutorial.
  • openni_kinect stack is installed.
    • To install it, follow this tutorial. You just need to do the Ubuntu installation. It consists of one easy step:
      sudo apt-get install ros-diamondback-openni-kinect
  • IPC-Bridge is installed.
    • If you installed the IPC-Bridge after 3/29/11, then you can move on to the tutorial to learn how to bringup your kinect and stream data into matlab:
      Using your Kinect
    • If you installed the IPC-bridge before 3/29/11 then you must follow the instructions below. The zip file has a bug in it, so we now we need to fix it.

Fixing your IPC-Bridge: Two small Changes

If you have installed the IPC-Bridge on your personal Ubuntu machine for Project 1, there are two changes that need to be made to make it work for Project2.

  1. Change the LD_LIBRARY_PATH - Some important libraries were moved around in Diamondback so we need to adjust the line that we added in our .bashrc for the IPC-Bridge to reflect those changes.
  2. Fix the ipc_sensor_msgs package - We only need the Image message type, however it currently causes central(IPC) to die because of a small bug.

How to Change your LD_LIBRARY_PATH:

  1. Open your .bashrc file.
  2. Remove the following line:
    export LD_LIBRARY_PATH=`rospack find ipc`/lib:/opt/ros/cturtle/ros/core/roscpp/lib/:/opt/ros/cturtle/ros/3rdparty/xmlrpcpp/lib/:/opt/ros/cturtle/ros/core/rosconsole/lib/:/opt/ros/cturtle/ros/core/roslib/lib/:/opt/ros/cturtle/ros/tools/rospack/lib/:${LD_LIBRARY_PATH}
  3. Add the following line:
    export LD_LIBRARY_PATH=`rospack find ipc`/lib:`rospack find roscpp`/lib:`rospack find roscpp_serialization`/lib:`rospack find xmlrpcpp`/lib:`rospack find rosconsole`/lib:`rospack find roslib`/lib:`rospack find rospack`/lib:`rospack find rostime`/lib:`rospack find cpp_common`/lib:${LD_LIBRARY_PATH}
  4. Close all your terminals and reopen them.

How to fix your broken ipc_sensor_msgs package:

Instead of reinstalling the ipc-bridge again (which is a valid solution), you can just re-download the ipc_sensor_msgs package and replace the one you have in your ipc_bridge_stack folder.

Follow these four easy steps:

  1. Go into your ipc_msgs stack folder:
    roscd ipc_msgs
  2. Delete the ipc_sensor_msgs folder.
  3. Download the new version of the ipc_sensor_msgs. Extract it into the ipc_msgs folder (where the old one used to be).
  4. Go into the new ipc_sensor_msgs folder and compile it:
    make

Once you have fixed your ipc_sensor_msgs package, you can continue on to:
Tutorial: Retrieving images from the Kinect using ROS into Matlab