In order to make your project or installation respond to user actions by using the data from camera, often you need to search, recognize, and track objects on the live video from camera. In this section, we will only cover searching objects.
There are several methods to search a specific object in an image:
This method can work with complex and overlapped objects but is sensitive to a number of conditions; for example, object should have distinctive edges and there should not be too many edges in the whole image.
This is the most advanced and robust technique today. Most famous algorithms are the Viola-Jones algorithm for searching frontal human faces and the HOG-method (Histogram of Orientated Gradients) for searching pedestrians, cars, bicycles, animals, and other outdoor objects.
We will consider a simple example of object searching with contour analysis. Here we will not search specific objects but will find all the bright objects in the image and find the centers of these objects. To achieve this, we will find contours and call the pixels inside each distinct contour, the object. Finding contours is provided by a functionality of the ofxCvContourFinder
class, which we will discuss now.
The ofxCvContourFinder
class is used for searching contours and bounding connected white regions in a binary image. These regions here are called blobs. The typical usage of ofxCvContourFinder
is as follows:
ofxCvContourFinder contourFinder;
object.contourFinder.findContours( mask, minArea, maxArea, maxNumber, findHoles );
Here mask
is a binary image. The parameters minArea
and maxArea
set the range of the pixels' number in the blobs to reject too small or too large blobs.
Also, maxNumber
is an upper limit of the number of resulted blobs and the findHoles
value specifies searching the holes mode. If it is false
, black holes in blobs are simply ignored, else the holes are regarded as blobs too.
For example, contourFinder.findContours( mask, 10, 10000, 20, false );
searches for not more than 20
white blobs containing 10
to 10000
pixels and the holes are ignored.
contourFinder.blobs
array. The number of blobs is given by contourFinder.blobs.size()
. Each contourFinder.blobs[i]
blob has the following members:area
– The number of pixels in the bloblength
– The perimeter of the blob's contourboundingRect
– The bounding rectangle of the blobcentroid
– The point of the blob's center of masshole
– A boolean value that is equal to true
if the blob is not a blob but the hole of some other blobpts
– The point array of the blob's contournPts
– The number of points in the contourNote that the list of blobs is calculated independently for each frame, so you should not assume that blobs with same the index i
mean the same blob on successive video frames. If you need to retain blobs identifiers, you should implement it in your own algorithm; for example, you can assign the IDs of the blobs in the current frame by getting the ID of the nearest blob in the previous frame.
contourFinder.draw( x, y, w, h )
function. Note that the function uses its internal colors for drawing and draws the blob's contour line and bounding box.Let's consider an example realizing the full processing of the input image image
to the list of objects' center's obj
.
Use the project Generator wizard for creating an empty project with the linked ofxOpenCv addon (see the Using ofxOpenCv section). Then, copy the fruits.mov
movie into bin/data
of the project, and copy sources of the example to the src
folder.
Here, we will consider just the part of the code related to searching for objects.
Assume that the scene has a dark background and the objects are brighter than the background. Then, the processing steps inside the update()
function will be the following:
ofxCvColorImage
image as follows:image.setFromPixels( video.getPixelsRef() );
imageDecimated
for storing the decimated image, and allocate it at the first iteration as follows:if ( !imageDecimated.bAllocated ) { imageDecimated.allocate( image.width * 0.5, image.height * 0.5 ); } imageDecimated.scaleIntoMe( image, CV_INTER_NN );
ofxCvGrayscaleImage grayImage
:grayImage = imageDecimated;
blurred = grayImage; blurred.blurGaussian( 9 );
if ( !background.bAllocated ) { background = blurred; }
absDiff()
) because we assume that objects are brighter than the background.diff = blurred; diff -= background;
mask = diff; mask.threshold( 40 );
Here the value 40
is the threshold parameter and should be adjusted for good results while using videos other than the one in example.
contourFinder
object of type ofxCvContourFinder
.contourFinder.findContours( mask, 10, 10000, 20, false );
This function searches no more than 20
white blobs containing 10
to 10000
pixels, and the holes are ignored.
obj
, namely, vector<ofPoint> obj
. For shortening the code, we use the reference to the blob list blobs
:vector<ofxCvBlob> &blobs = contourFinder.blobs; int n = blobs.size(); //Get number of blobs obj.resize( n ); //Resize obj array for (int i=0; i<n; i++) { obj[i] = blobs[i].centroid; //Fill obj array }
Compile and run the project. You will see an animation with four images showing the processing video with rolling fruits. The first image is a frame from the video, which is decimated by 50 percent. The second image is the difference diff
between the smoothed and background images. The third image is a thresholded image mask
with contours drawn over it. Finally, the last image is the original (decimated) frame with crosses marking the found objects.
Note that the difference image does not contain the bright spot that existed at the bottom-left corner of the original image, because this spot is included in the background image and hence was subtracted.
Press 2 and you will see an example of using the centers of the objects obj
for generating images.
This is the original image with some white lines drawn over it. These lines depict the isolines of some function f(x,y), which depend on the obj
array. Each object's center obj[i]
adds to the function cone with the center obj[i]
. See a detailed description of this function in generateImg()
function in the full example's code. For returning to the processing screen, press 1.
In this example, we were interested in searching all the objects irrespective of their shape. Nevertheless, such a simple example can be used in a wide range of projects for detecting objects such as bright spots. In particular, it is useful in the following situations:
Note that for accurately detecting the fact that a foot is standing on the floor, it is often better to use depth cameras; see how to do this in the Creating interactive surface section in Chapter 10, Using Depth Cameras. Though depth cameras are much simpler to adjust and use, they have limitations in their tracking range. So three or more depth cameras are needed for tracking areas of size 10×10 meters. And when using ordinary cameras, just one or two cameras can be enough.
Although the ofxOpenCv addon provides a handy interface for basic filtering and geometrical transformations of images, it is a very small part of OpenCV. So now we will learn how to use other OpenCV functions in your projects.