In this chapter, we worked only with objects in screen coordinates, by just using depth values of depth images. But in many applications, you need to use the 3D point cloud, which represents the depth image points in 3D space—in the world coordinate system centered at the camera. For obtaining a 3D point cloud from the depth image, you should convert all its pixels to 3D points using the openNIDevice.projectiveToWorld( p )
function. Namely, each pixel (x
, y
) should be represented as point p
, where p.x = x
, p.y = y
, and p.z
is equal to the value of the depth image's pixel (x
, y
): p.z = openNIDevice.getDepthRawPixels()[ x + w * y ]
.
The 3D point cloud can be used for:
Other topics to learn are user tracking, hand tracking, and gesture recognition. We suggest that you do this not just by exploring the ofxOpenNI source codes, but also by studying the original OpenNI library.