Motion template is a technique in image processing for finding a small part of an image or silhouette that matches a template image. This template matcher is used to make comparisons with respect to similarity and to examine the likeness or difference. Templates might potentially require sampling of a large number of points. However, it is possible to reduce these numbers of points by reducing the resolution of the search; another technique to improve these templates is to use pyramid images.
In OpenCV's examples ([opencv_source_code]/samples/c/motempl.c), a related program can be found.
We now assume that we have a good silhouette or template. New silhouettes are then captured and overlaid using the current time stamp as the weight. These sequentially fading silhouettes record the history of the previous movement and are thus referred to as the Motion history template. Silhouettes whose time stamp is more than a specified DURATION older than the current time stamp are set to zero. We have created a simple example (motionHistory) using the updateMotionHistory()OpenCV function on two frames as follows:
void updateMotionHistoryTemplate(Mat& prev_frame, Mat& frame, Mat& history) { //1-Calculate the silhouette of difference between the two //frames absdiff(frame, prev_frame, prev_frame); //2-Applying a threshold on the difference image double threshold_val = 100; threshold(prev_frame,prev_frame,threshold_val,255,THRESH_BINARY); //3-Calculate the current time clock_t aux_time = clock(); double current_time = (aux_time-INITIAL_TIME)/CLOCKS_PER_SEC; //4-Performing the Update Motion history template updateMotionHistory(prev_frame, history, current_time, DURATION); }
This example shows you a window where the motion history is drawn. The first step is to obtain a silhouette; the Background subtraction technique is used for this. The difference in the absolute value is obtained from the two input frames. In the second step, a binary thresholding is applied to remove noise from the silhouette. Then, the current time is obtained. The final step is to perform the updating of the Motion history template using OpenCV's function.
We have also set DURATION to 5. Note that it is necessary to initialize INITIAL_TIME and history. Besides, we can use this function call from the modified videoCamera example as follows:
...
// Calculate the initial time
INITIAL_TIME = clock()/CLOCKS_PER_SEC;
//Create a Mat to save the Motion history template
Mat history(prev_frame.rows, prev_frame.cols, CV_32FC1);
while(!finish)
{
capture.read(frame);
cvtColor(frame,frame,COLOR_BGR2GRAY);
// Using Update Motion history template
updateMotionHistoryTemplate(prev_frame, frame, history);
imshow("Video Camera", history);
...To use the clock() function, which gets the current time, we need to include <ctime>. Some screen captures will be shown in which a person is walking in front of the camera.
Output of the motionHistory example
Once the Motion templates have a collection of object silhouettes overlaid in time, we can obtain the directions of movement by computing the gradients of the history image. The following example (motionGradient) computes the gradients:
void motionGradientMethod(Mat& history, Mat& orientations) { //1-Set the parameters double max_gradient = 3.0; double min_gradient = 1.0; //The default 3x3 Sobel filter int apertura_size = 3; //Distance to show the results int dist = 20; Mat mask = Mat::ones(history.rows, history.cols, CV_8UC1); //2-Calcule motion gradients calcMotionGradient(history, mask, orientations, max_gradient, min_gradient, apertura_size); //3-Show the results Mat result = Mat::zeros(orientations.rows, orientations.cols, CV_32FC1); for (int i=0;i<orientations.rows; i++) { for (int j=0;j<orientations.cols; j++) { double angle = 360-orientations.at<float>(i,j); if (angle!=360) { Point point_a(j, i); Point point_b(round(j+ cos(angle)*dist), round(i+ sin(angle)*dist)); line(result, point_a, point_b, Scalar(255,0,0), 1); } } } imshow("Result", result); }
A screen capture is shown with a person moving his head in front of the camera (see the following screenshot). Each line represents the gradient for each pixel. Different frames also overlap at a t time:
Output of the motionGradient example (a person is moving his head in front of the camera).
The preceding example shows you a window that displays the directions of movement. As the first step, the parameters are set (the maximum and minimum gradient value to be detected). The second step uses the calcMotionGradient() function to obtain a matrix of the gradient direction angles. Finally, to show the results, these angles are drawn on the screen using a default distance, which is dist. Again, we can use this function from the following modified videoCamera example:
... //Create a Mat to save the Motion history template Mat history(prev_frame.rows, prev_frame.cols, CV_32FC1); while(!finish) { capture.read(frame); cvtColor(frame,frame,COLOR_BGR2GRAY); //Using Update Motion history template updateMotionHistoryTemplate(prev_frame, frame, history); //Calculate motion gradients Mat orientations = Mat::ones(history.rows, history.cols, CV_32FC1); motionGradientMethod(history, orientations); ...