The example we saw previously is a very simple one. The background is still, there is no particular fluctuation of the brightness on the scene, and the moving object has a very distinctive color. Let's now see what happens when we have a more complex scene to handle. We will work with the driving video, presented in the previous chapter, which represents one of the most challenging scenarios; a moving camera, with a scene with moving background, and several moving objects. Let's start:
- Our first step is, as always, to load the video we will process:
>> obj = VideoReader('inter.avi'), % Read video file >> vid = read(obj); % Load all frames - The montage of our frames looks like this:
>> montage(vid,'Size',[4 7]) % Using a 4x7 grid for our 28 frames
- Now, we will subtract each frame from the next one, excluding the first. The result will be stored in a predefined matrix:
>> subtracted = zeros(size(vid)); % Preallocate space >> for i = 2:size(vid,4), % For all frames but the first one % Subtract each frame from its next one subtracted(:,:,:,i) = vid(:,:,:,i) - vid(:,:,:,i-1); end
- Let's see the montage of all the motion detection frames:
>> montage(vid,'Size',[4 7]) % Using a 4x7 grid for our 28 frames
- Now, let's add all the frames together and see what comes up. This time we will do it for both the original video and the motion detection one:
>> total1 = sum(vid,4); % Addition of all frames (4-th dim.) >> total1 = mat2gray(total1); % Normalize the whole matrix >> total2 = sum(subtracted,4); % Addition of all frames >> total2 = mat2gray(total2); % Normalize the whole matrix >> subplot(1,2,1),imshow(total1),title('Sum of originalframes') >> subplot(1,2,2),imshow(total2),title('Sum of detection frames')
This time around, our results are not so informative. While the subtraction results showed the moving pixels in each pair of consecutive frames, there was no real indication of which of them was moving, or were still and just have a relative motion to the moving camera. Even more vague results were produced by the addition of the frames. Both the results of adding the original frames and of adding the subtracted frames were blurry and do not give us enough information about the motion in the scene.
The only areas of the image that appear to have stayed still during the short duration of the video were the sky and a small portion of the road. However, this result is highly unintuitive, since the road surface was definitely moving relative to our camera. This, in fact, is one of the main difficulties in detecting motion in videos. Smooth objects with very little detail in them appear motionless, except for their borders (when they are colored differently to the background). When the object under examination is the road (or something of similarly large size and smoothness), a big part of it appears to be still, according to the frame subtraction result. Another difficulty in detecting motion in a video is posed by the insufficient frame rate for very quick moving objects, that is, when the objects in the video make large movements from one frame to the next, motion detection might produce a very evident ghosting effect.
Now that you have seen the difference in difficulties between the still scenes and complex scenes, let's go back to the easy problem. You should be able to write a piece of code that receives input from a camera, converts the frames to grayscale, and displays a warning message every time the mean difference of brightness values between the two consecutive frames exceeds a predefined level.
For a more complicated example, you could even split your frame in rectangular areas and announce in the warning message which area has been intruded (for example, by using a command, such as disp('Area #1 appears to have high activity!')).
All the tools you need to accomplish this task have been presented. You should revise how you can get a live video input in MATLAB, convert RGB to grayscale, and calculate mean values in two dimensions (Hint: the function mean2 is useful). For the second task, you have to devise a way to split the image into blocks and check all of them with a for loop.