In this chapter, we will show you different techniques related to motion, as estimated from video frames. After a short introduction and definitions, we will show you how to read video frames captured from a camera. Then, we will tackle the all-important Optical Flow technique. In the third section, we will show you different functions that can be used for tracking. The Motion history and Background subtraction techniques are explained in the fourth and fifth sections, respectively. Finally, image alignment with the ECC method is explained. Every example has been developed and tested for the latest version of OpenCV in GitHub. Most of the functions can work in the previous versions equally, leading to some changes that will be discussed. Most of the functions introduced in this chapter are in the video module.
Note
To test the latest source code available in GitHub, go to https://github.com/itseez/opencv and download the library code as a ZIP file. Then unzip it to a local folder and follow the same steps described in Chapter 1, Getting Started, to compile and install the library.
Motion is a very important topic in Computer Vision. Once we detect and isolate an object or person of interest, we can extract valuable data such as positions, velocity, acceleration, and so on. This information can be used for action recognition, behavior pattern studies, video stabilization, augmented reality, and so on.
The Optical Flow technique is a pattern of an object's apparent motion. Surfaces and edges in a visual scene are caused by relative motion between an observer and scene or between the camera and the scene. The concept of the Optical Flow technique is central in Computer Vision and is associated with techniques/tasks such as motion detection, object segmentation, time-to-control information, focus of expansion calculations, luminance, motion compensated encoding, and stereo disparity measurement.
Video tracking consists of locating a moving object (or multiple objects) over time using videos captured from a camera or file. The aim of video tracking is to associate target objects in consecutive video frames. It has a variety of uses, some of which are video editing, medical imaging, traffic control, augmented reality, video communication and compression, security and surveillance, and human-computer interaction.
Motion templates were invented at the MIT Media Lab by Bobick and David in 1996. The use of the motion templates is a simple yet robust technique that tracks general movement. OpenCV motion template functions only work with single channel images. A silhouette (or part of a silhouette) of an object is needed. These silhouettes can be obtained in different ways. For example, segmentation techniques can be used to detect the interest object and then perform tracking with motion templates. Another option is to use the Background subtraction technique to detect foreground objects and then track them. There are other techniques too, although, in this chapter, we will see two examples that use the Background subtraction technique.
Background subtraction is a technique by which an image foreground or region of interest is extracted for further processing, for example, people, cars, text, and so on. The Background subtraction technique is a widely used approach for detecting moving objects in videos captured from static cameras. The essence of the Background subtraction technique is to detect the moving objects from differences between current frames and a reference image taken without target objects present, which is usually called a background image.
Image alignment can be seen as a mapping between the coordinate systems of two or more images taken from different points of view. The first step is, therefore, the choice of an appropriate geometric transformation that adequately models this mapping. This algorithm can be used in a wide range of applications, such as image registration, object tracking, super-resolution, and visual surveillance by moving cameras.