In this example, we will make a loop for continuous acquisition and per channel contrast adjustment of frames from our camera. We will time the process using the profile function of MATLAB so that we demonstrate the bottleneck of our process. This way, you can get an idea of the time issues arising when performing video processing in MATLAB. Bear in mind that the machine used for these experiments had a Q9550 Quad-Core CPU, at 2.8 GHz:
- First off, we get our hardware ready:
>> imaqreset >> vid = videoinput('winvideo', 1, 'dvsd_720x576'), - Now, we preallocate space for the matrix that will hold our frames. For our experiment, 100 frames will be enough:
>> test = uint8(zeros(576,720,3,100));
- We then start the MATLAB profiler, which will analyze the time spent on each of our functions:
>>profile on
- Next, we write the
forloop that will make it all happen:>> for i = 1:100 temp = getsnapshot(vid); % Acquire a frame fprintf('Processing frame number %d… ',i) % Announcement test(:,:,i) = imadjust(temp); % Adjust contrast subplot(1,2,1),imshow(temp ) % Display current frame subplot(1,2,2),imshow(test(:,:,:,i)) % Display processed frame endEach display of the resulting frames looks like the following screenshot:
- Now, we close the profiler and display its results:
>> profile off % Close profiler >> profile viewer % Display profiling results
These were the profiling results. Each of the functions was analyzed with regards to the number of times it was called, the total time spent on it, and its self-time that is the time spent in the core of the functions, disregarding its child functions.
This example gave us an idea of the time issues hiding behind video processing tasks, even if they are very basic. The process we selected to investigate was a continuous acquisition of frames, followed by a per-channel contrast enhancement using the imadjust function (as discussed in previous chapters). The profile function provided by MATLAB was used to inspect the timing details of each step of our process. By analyzing our results, we easily concluded that getsnapshot is the bottleneck of our whole process, taking a total of 26.824 seconds of the time. Taking into account that the second most time-consuming function was imshow (4.349 seconds in total), we understood that acquiring the snapshots was unreasonably time-consuming.
The results of the previous example were a huge disappointment. Needing to spend approximately 27 seconds only for the image acquisition part of our 100 frames acquisition program is prohibiting for real-time applications. It means that each acquisition takes approximately 27/100 = 0.27 seconds, leading to a frame rate of about 1/0.27 = 3.7 fps just for the acquisition. This is very far from the 25 fps goal that we need for our PAL video processing applications. It may be enough for time-lapses, but by no means does it fit the real-time video processing requirements.
In this case, the solution to our problem is surprisingly unintuitive and simple. The reason why the getsnapshot function takes so much time lies in the way it works. Ideally, it needs to have a Preview window of the acquired video open, in order to acquire the frames faster. If it does not, then the function is delayed because it tries to generate the preview and grab the frame silently. Let's try to resolve this issue.