The Kinect is a device being sold by Microsoft as an accessory for the XBox 360. It detects and tracks the various parts of the human body to use as a controller for video games.

The Kinect uses two cameras – a normal photo camera and one that gauges depth by projecting an array of infrared dots into the room and analyzing the changes to the pattern.  Effectively, it’s a 3D scanner. For $140. Which is awesome.

Adafruit announced a $3000 prize for anyone who could reverse engineer the Kinect and release open source drivers for it. The prize was successfully claimed within days. The resulting driver was called OpenKinect, and it allowed hackers to start integrating the Kinect into a variety of programs, such as Processing.

The applications are endless. Computer vision. Interactive video art. Independent games. Music.

Interactive Puppet Prototype with Xbox Kinect from Theo Watson on Vimeo.

Imagine being able to start and stop audio playback by tapping your foot. Or controlling pitch and volume by waving your hands in the air. Kind of a virtual Theremin, but much more flexible.

Microsoft objected to this Kinect hacking at first, but it turns out they didn’t have much right to be upset. The device and its drivers were developed by PrimeSense, which licensed the technology (non-exclusively) to Microsoft. They call it PrimeSensor, and they decided to release the drivers, which they call OpenNI. They have a middleware layer called NITE which does all the cool limb and joint tracking.

There is a utility called OSCeleton which does the body tracking of the Kinect, and sends the data as OSC messages, which can then be received by Pure Data and used as input for audio and video generation.