You always need to initialize an image before using it for the first time. Let us look at a few functions used to initialize images:

There are two functions that are close to the initialization stage:

Algebraic operations apply mathematical operations such as addition and subtraction to each pixel of the images. The following are a few algebraic functions:

All the image types except ofxCvFloatImage have a limited range of pixel values (see the discussion of ranges in the preceding information box for the operator =). In the case when the result of any operation goes beyond the range, the saturation arithmetic is applied, that is, the value is truncated to the range. For example, for an image of type ofxCvGrayscaleImage, the values 300 and -10 will be truncated to 255 and 0 respectively. Hence, if you perform advanced mathematical calculations with images, it is a good idea to convert input images to ofxCvFloatImage first, then perform calculations, and finally convert the final result to the required type.

The drawing functions are similar to the corresponding functions of the ofImage class for openFrameworks' images, discussed in Chapter 4, Images and Textures. The following are a few drawing functions:

For implementing custom image processing functions or using OpenCV functions not implemented in the ofxOpenCv addon, you need to have access to the pixel values and OpenCV image inside the ofxCv image. There are number of functions for dealing with it:

There are a number of functions for manipulating color planes. They are as follows:

The class ofxCvColorImage has two functions for converting between RGB (Red, Green, Blue) and HSV (Hue, Saturation, Value) color spaces: convertRgbToHsv() and convertHsvToRgb().

Now we will consider an example of using ofxCv images for a simple motion detector.

Let's consider a live video from the camera or a video from a movie, and consider the absolute difference between its two successive frames, which is computed using the function grayImage.absDiff( grayImage2 ), considered in the Algebraic operations with images section. The regions in this difference image, corresponding to the moving objects, will have higher values than the static regions. So, for getting pixels with high values, it is possible to detect the regions of motion in the video. This information can be used for controlling the behavior of your application; for example, if you have a particle system, the motion areas can be used as places of particles' emitting or as areas of particles' attraction. Then, people walking in front of your camera will see how their silhouette controls the particles on the screen.

This method of using difference image is simple and has been successfully used in a number of interactive projects for more than thirty years now. However, if you consider two successive difference images, they will most likely have very few common pixels with high values. The reason is that difference image emphasizes the changed pixels of successive frames, or in another words, "motion border", which changes each frame. So difference image is not stable in time. To regularize it, it is a good idea to accumulate the differences in an image buffer that slowly decreases its values.

The following example illustrates the calculation of absolute differences and accumulation of them in the buffer using ofxCv images of the ofxOpenCv addon.

Use the Project Generator for creating an empty project with the linked ofxOpenCv addon (see the Using ofxOpenCv section). Then, copy the handsTrees.mov movies into the bin/data folder of the project.

Include the addon's header into the testApp.h file, just after the #include "ofMain.h" line:

#include "ofMain.h"
#include "ofxOpenCv.h"

Also, add the following lines in the testApp class declaration:

ofVideoPlayer video;         //Declare the video player object

ofxCvColorImage image;	      //The current video frame

//The current and the previous video frames as grayscale images
ofxCvGrayscaleImage grayImage, grayImagePrev;

ofxCvGrayscaleImage diff;    //Absolute difference of the frames
ofxCvFloatImage diffFloat;   //Amplified difference images
ofxCvFloatImage bufferFloat; //Buffer image

Now let's assemble testApp.cpp. The testApp::setup() function loads and starts the video as follows:

void testApp::setup(){
  video.loadMovie( "handsTrees.mov" );  //Load the video file
  video.play();                       //Start the video to play
}

The testApp::update() function reads video frames and processes them by calculating the absolute difference diff, amplifying it for better visibility in diffFloat, and then updating the accumulate buffer bufferFloat. Note, we check the fact that an image is initialized using the bAllocated value:

void testApp::update(){
  video.update();  //Decode the new frame if needed
  //Do computing only if the new frame was obtained
  if ( video.isFrameNew() ) {
      //Store the previous frame, if it exists till now
      if ( grayImage.bAllocated ) {
          grayImagePrev = grayImage;
      }

      //Getting a new frame
      image.setFromPixels( video.getPixelsRef() );
      grayImage = image;  //Convert to grayscale image

      //Do processing if grayImagePrev is inited
      if ( grayImagePrev.bAllocated ) {
          //Get absolute difference
          diff.absDiff( grayImage, grayImagePrev );

          //We want to amplify the difference to obtain
          //better visibility of motion
          //We do it by multiplication. But to do it, we
          //need to convert diff to float image first
          diffFloat = diff;   //Convert to float image
          diffFloat *= 5.0;   //Amplify the pixel values

          //Update the accumulation buffer
          if ( !bufferFloat.bAllocated ) {
              //If the buffer is not initialized, then
              //just set it equal to diffFloat
              bufferFloat = diffFloat;
          }
          else {
              //Slow damping the buffer to zero
              bufferFloat *= 0.85;
              //Add current difference image to the buffer
              bufferFloat += diffFloat;
          }
      }
  }
}

Finally, the testApp::draw() function draws four images (from left to right and from top to bottom). These four images are as follows:

Let's take a look at the code:

void testApp::draw(){
  ofBackground( 255, 255, 255 );  //Set the background color

  //Draw only if diffFloat image is ready.
  //It happens when the second frame from the video is obtained
  if ( diffFloat.bAllocated ) {
      //Get image dimensions
      int w = grayImage.width;
      int h = grayImage.height;

      //Set color for images drawing
      ofSetColor( 255, 255, 255 );

      //Draw images grayImage,  diffFloat, bufferFloat
      grayImage.draw( 0, 0, w/2, h/2 );
      diffFloat.draw( w/2 + 10, 0, w/2, h/2 );
      bufferFloat.draw( 0, h/2 + 10, w/2, h/2 );

      //Draw the image motion areas

      //Shift and scale the coordinate system
      ofPushMatrix();
      ofTranslate( w/2+10, h/2+10 );
      ofScale( 0.5, 0.5 );

      //Draw bounding rectangle
      ofSetColor(0, 0, 0);
      ofNoFill();
      ofRect( -1, -1, w+2, h+2 );

      //Get bufferFloat pixels
      float *pixels = bufferFloat.getPixelsAsFloats();
      //Scan all pixels
      for (int y=0; y<h; y++) {
          for (int x=0; x<w; x++) {
              //Get the pixel value
              float value = pixels[ x + w * y ];
              //If value exceed threshold, then draw pixel
              if ( value >= 0.9 ) {
                  ofRect( x, y, 1, 1 );
                  //Rectangle with size 1x1 means pixel
                  //Note, this is slow function, 
                  //we use it here just for simplicity
              }
          }
      }
      ofPopMatrix();	//Restore the coordinate system
  }
}

Run the example. You will see an animation consisting of four images on the screen: the current movie frame in grayscale, absolute difference of the current and the previous frame, the accumulated buffer, and finally, the motion areas shown in black color:

You can observe that all the operations in the example can be done using pixel-by-pixel processing, described in Chapter 5, Working with Videos. So, why use complicated stuff like OpenCV? The answer is that although the example is very simple, when you do more complicated image processing and then use pixel-by-pixel programming, it makes the code cumbersome. When using OpenCV, you can do most of the image operations with a single line of code. Also, OpenCV uses various optimizations, so using it usually improves the performance of your project.

Now we will consider the control parameters of the motion detection algorithm.

There are three parameters of the motion detection algorithm:

The second and third images on the screen are the result of drawing float-valued images diffFloat and bufferFloat. As we discussed earlier, pixel values of these images are mapped from the range [0, 1] to 0 to 255 while drawing. So, all the pixel values greater than 1.0 are rendered in white color. In our case, pixels of the bufferFloat image can have values greater than 1.0, so its image on the screen is clamped in the sense of color representation. To reduce the color clamping on the screen, decrease the amplification parameter.

It is easy to change the previous example to search motion areas in a live video from a camera. That is, find the following line in testApp.h:

ofVideoPlayer video;  //Declare the video player object

Replace the preceding line with the following:

ofVideoGrabber video;  //Video grabber from the camera

Now, find the following lines in testApp.cpp in the testApp::setup() function:

  video.loadMovie( "handsTrees.mov" );  //Load the video file
  video.play();                       //Start the video to play

Replace the preceding lines with the following:

  video.initGrabber( 640, 480 ); //Start the video grabber

When you run the example with a live video, you will possibly find that the motion has not tracked so well as in the prerecorded video handsTrees.mov. In this case, you need to adjust the algorithm's parameters, see the Discussing the algorithm's parameters section.

We have considered the algorithm of motion detection, which is great for learning the basics of computer vision and making simple interactive installations. Though, it has a drawback in that you need to adjust its parameters depending on the light conditions. Also, the result of detection depends on the colors of the objects.

There are several ways to avoid this problem:

Now we will consider the methods of image filtering, which include smoothing and other essential image processing operations.