Now it is time to implement the process described in the previous paragraph. The function that we will use is based on the previous two examples:
function [vid] = SpatioTemporalAverage(vid,order)
% Function for de-interlacing a video using spatiotemporal averaging
% Inputs:
% vid – Input video matrix (we assume color video)
% order - Choice for row replacement
% Output:
% vid - Output video matrix (de-interlaced)
vid = single(vid); % Convert matrix to single to perform averaging
switch order % Checking choice for the order of merging
case 1 % Odd rows from odd frames
for fr = 2:size(vid,4)-1 % For all frames (but first&last)
for row = 2:size(vid,1)-1 % For all rows (but first&last)
if mod(fr,2) == 0 % For even frames
if mod(row,2) == 0 % Replace even rows
vid(row,:,:,fr) = ...((vid(row,:,:,fr-1) + vid(row,:,:,fr+1)) ...+ (vid(row-1,:,:,fr) + vid(row+1,:,:,fr))) / 4;
end
else % For odd frames
if mod(row,2) ~= 0 % Replace odd rows
vid(row,:,:,fr) = ...((vid(row,:,:,fr-1) + vid(row,:,:,fr+1)) ...+ (vid(row-1,:,:,fr) + vid(row+1,:,:,fr))) / 4;
end
end
end
end
case 2 % Even rows from odd frames
for fr = 2:size(vid,4)-1 % For all frames (but first&last)
for row = 2:size(vid,1)-1 % For all rows (but first&last)
if mod(fr,2) == 0 % For even frames
if mod(row,2) ~= 0 % Replace odd rows
vid(row,:,:,fr) = ...((vid(row,:,:,fr-1) + vid(row,:,:,fr+1)) ...+ (vid(row-1,:,:,fr) + vid(row+1,:,:,fr))) / 4;
end
else % For odd frames
if mod(row,2) == 0 % Replace even rows
vid(row,:,:,fr) = ...((vid(row,:,:,fr-1) + vid(row,:,:,fr+1)) ...+ (vid(row-1,:,:,fr) + vid(row+1,:,:,fr))) / 4;
end
end
end
end
otherwise
disp('Unknown method.') % Error message
end
vid = uint8(vid); % Convert matrix back to uint8At this point, we can check our results to evaluate the importance of including spatiotemporal information for the deinterlacing:
- First, we load the same video as before into the Workspace window and read in all of its frames:
>> obj = VideoReader('inter.avi'), >> vid = read(obj); - Now, we use the new function, once with the second input as
1, and once with the second input as2:>>[vid1] = SpatioTemporalAverage(vid,1); >>[vid2] = SpatioTemporalAverage(vid,2);
- Now, we demonstrate the results for an odd and an even frame:
>> subplot(2,3,1),imshow(vid(:,:,:,5)),title('Odd frame') >> subplot(2,3,4),imshow(vid(:,:,:,6)),title('Even frame') >> subplot(2,3,2),imshow(vid3(:,:,:,5)),title('Odd frame-1') >> subplot(2,3,3),imshow(vid4(:,:,:,5)),title('Odd frame-2') >> subplot(2,3,5),imshow(vid3(:,:,:,6)),title('Even frame-1') >> subplot(2,3,6),imshow(vid4(:,:,:,6)),title('Even frame-2')
Once again, the function is based on previous examples, with only few alterations, highlighted in the previous code. The first alteration was the frames that were treated by the process. Instead of leaving only the last one out, here we also left the first one out. Similarly, we did not process both the first and the last rows of each frame. The last alteration was the interpolation rule, which became spatiotemporal (spatial because rows of the same frame were weighed-in, and temporal because rows from the next and previous frames were also weighed-in):
vid(row,:,:,fr) = ...((vid(row,:,:,fr-1) + vid(row,:,:,fr+1)) ...+ (vid(row-1,:,:,fr) + vid(row+1,:,:,fr))) / 4;
This way, the resulting frame carried information both from its contents and from the neighboring frames. To demonstrate our results, we loaded and processed the same video as before, using steps 1 through 3. If you compare the results generated now to those generated in the previous example, you will clearly understand the importance of the spatiotemporal technique.
Now it is your turn to examine and evaluate the deinterlacing techniques presented so far. The goal is to apply these methods to interlaced videos of your choice and crop the frames accordingly so that you compare details of each method. Try to use both the still scene videos and high motion videos.
Note
As we saw in this section, scenes with a lot of motion are not handled completely effectively by the methods presented here. In these cases, to further reduce artifacts, we have to consider using more complex methods of interpolation that are motion-adaptive or motion-compensated. These methods are quite complex for the purposes of this book and require more sophisticated algorithms, with somewhat difficult mathematics.