Creating a window with cv::namedWindow()

First, we want to be able to create a window and show an image on the screen using HighGUI. The function that does the first part for us is cv::namedWindow(), and it expects a name for the new window and one flag. The name appears at the top of the window, and is also used as a handle for the window that can be passed to other HighGUI functions.² The flag argument indicates whether the window should autosize itself to fit an image we put into it. Here is the full prototype:

int cv::namedWindow(
  const string&  name,                    // Handle used to identify window
  int            flags = 0                // Used to tell window to autosize
);

For now, the only valid options available for flags are to set it to 0 (the default value), which indicates that users are able (and required) to resize the window, or to set it to cv::WINDOW_AUTOSIZE.³ If cv::WINDOW_AUTOSIZE is set, then HighGUI resizes the window to fit automatically whenever a new image is loaded, but users cannot resize the window.

Once we create a window, we usually want to put something inside it. But before we do that, let’s see how to get rid of the window when it is no longer needed. For this, we use cv::destroyWindow(), a function whose only argument is a string: the name given to the window when it was created.

int cv::destroyWindow(
  const string&  name,                    // Handle used to identify window
);

Drawing an image with cv::imshow()

Now we are ready for what we really want to do: load an image and put it into the window where we can view it and appreciate its profundity. We do this via one simple function, cv::imshow():

void cv::imshow(
  const string&  name,                    // Handle used to identify window
  cv::InputArray image                    // Image to display in window
);

The first argument is the name of the window within which we intend to draw. The second argument is the image to be drawn. Note that the window will keep a copy of the drawn image and will repaint it as needed from that buffer, so a subsequent modification of the input image will not change the contents of the window unless a subsequent call to cv::imshow() is made.

Updating a window and cv::waitKey()

The function cv::waitKey() is to wait for some specified (possibly indefinite) amount of time for a keypress on the keyboard, and to return that key value when it is received. cv::waitKey() accepts a keypress from any open OpenCV window (but will not function if no such window exists).

int cv::waitKey(
  int delay = 0                     // Milliseconds until giving up (0='never')
);

cv::waitKey() takes a single argument, delay, which is the amount of time (in milliseconds) for which it will wait for a keypress before returning automatically. If the delay is set to 0, cv::waitKey() will wait indefinitely for a keypress. If no keypress comes before delay milliseconds has passed, cv::waitKey() will return –1.

There is a second, less obvious function of cv::waitKey(), which is to provide an opportunity for any open OpenCV window to be updated. This means that if you do not call cv::waitKey(), your image may never be drawn in your window, or your window may behave strangely (and badly) when moved, resized, or uncovered.⁴

An example displaying an image

Let’s now put together a simple program that will display an image on the screen (see Example 9-1). We can read a filename from the command line, create a window, and put our image in the window in 15 lines (including comments!). This program will display our image as long as we want to sit and appreciate it, and then exit when the Esc key (ASCII value of 27) is pressed.

int main( int argc, char** argv ) {

  // Create a named window with the name of the file
  //
  cv::namedWindow( argv[1], 1 );

  // Load the image from the given filename
  //
  cv::Mat = cv::imread( argv[1] );

  // Show the image in the named window
  //
  cv::imshow( argv[1], img );

  // Idle until the user hits the Esc key
  //
  while( true ) {
    if( cv::waitKey( 100 /* milliseconds */ ) == 27 ) break;
  }

  // Clean up and don't be piggies
  //
  cv::destroyWindow( argv[1] );

  exit(0);
}

For convenience, we have used the filename as the window name. This is nice because OpenCV automatically puts the window name at the top of the window, so we can tell which file we are viewing (see Figure 9-1). Easy as cake.

Figure 9-1. A simple image displayed with cv::imshow()

Before we move on, there are a few other window-related functions you ought to know about. They are:

void cv::moveWindow( const char* name, int x, int y );
void cv::destroyAllWindows( void );
int  cv::startWindowThread( void );

cv::moveWindow() simply moves a window on the screen so that its upper-left corner is positioned at pixel location: x, y. cv::destroyAllWindows() is a useful cleanup function that closes all of the windows and deallocates the associated memory.

On Linux and Mac OS X, cv::startWindowThread() tries to start a thread that updates the window automatically, and handles resizing and so forth. A return value of 0 indicates that no thread could be started—for example, because there is no support for this feature in the version of OpenCV that you are using. Note that, if you do not start a separate window thread, OpenCV can react to user interface actions only when it is explicitly given time to do so (this happens when your program invokes cv::waitKey()).

Mouse events

Now that we can display an image to a user, we might also want to allow the user to interact with the image we have created. Since we are working in a window environment and since we already learned how to capture single keystrokes with cv::waitKey(), the next logical thing to consider is how to “listen to” and respond to mouse events.

Unlike keyboard events, mouse events are handled by a more traditional callback mechanism. This means that, to enable response to mouse clicks, we must first write a callback routine that OpenCV can call whenever a mouse event occurs. Once we have done that, we must register the callback with OpenCV, thereby informing OpenCV that this is the correct function to use whenever the user does something with the mouse over a particular window.

Let’s start with the callback. For those of you who are a little rusty on your event-driven program lingo, the callback can be any function that takes the correct set of arguments and returns the correct type. Here, we must be able to tell the function to be used as a callback exactly what kind of event occurred and where it occurred. The function must also be told if the user was pressing such keys as Shift or Alt when the mouse event occurred. A pointer to such a function is called a cv::MouseCallback. Here is the exact prototype that your callback function must match:

void your_mouse_callback(
  int   event,                        // Event type (see Table 9-1)
  int   x,                            // x-location of mouse event
  int   y,                            // y-location of mouse event
  int   flags,                        // More details on event (see Table 9-1)
  void* param                         // Parameters from cv::setMouseCallback()
);

Now, whenever your function is called, OpenCV will fill in the arguments with their appropriate values. The first argument, called the event, will have one of the values shown in Table 9-1.

The second and third arguments will be set to the x- and y-coordinates of the mouse event. Note that these coordinates represent the pixel coordinates in the image independent of the other details of the window.⁵

The fourth argument, called flags, is a bit field in which individual bits indicate special conditions present at the time of the event. For example, cv::EVENT_FLAG_SHIFTKEY has a numerical value of 16 (i.e., the fifth bit, or 1<<4); so, if we wanted to test whether the Shift key were down, we could simply compute the bitwise AND of flags & cv::EVENT_FLAG_SHIFTKEY. Table 9-2 shows a complete list of the flags.

The final argument is a void pointer that can be used to have OpenCV pass additional information, in the form of a pointer, to whatever kind of structure you need.⁶

Next, we need the function that registers the callback. That function is called cv::setMouseCallback(), and it requires three arguments.

void cv::setMouseCallback(
  const string&     windowName,         // Handle used to identify window
  cv::MouseCallback on_mouse,           // Callback function
  void*             param     = NULL    // Additional parameters for callback fn.
);

The first argument is the name of the window to which the callback will be attached; only events in that particular window will trigger this specific callback. The second argument is your callback function. The third argument, param, allows us to specify the param information that should be given to the callback whenever it is executed. This is, of course, the same param we were just discussing with the callback prototype.

In Example 9-2, we write a small program to draw boxes on the screen with the mouse. The function my_mouse_callback() responds to mouse events, and it uses the events to determine what to do when it is called.

#include <opencv2/opencv.hpp>

// Define our callback which we will install for
// mouse events
//
void my_mouse_callback(
   int event, int x, int y, int flags, void* param
);

Rect box;
bool drawing_box = false;

// A little subroutine to draw a box onto an image
//
void draw_box( cv::Mat& img, cv::Rect box ) {
  cv::rectangle(
    img,
    box.tl(),
    box.br(),
    cv::Scalar(0x00,0x00,0xff)    /* red */
  );
}

void help() {
  std::cout << "Call: ./ch4_ex4_1
" <<
    " shows how to use a mouse to draw regions in an image." << std::endl;
}

int main( int argc, char** argv ) {

  help();
  box = cv::Rect(-1,-1,0,0);
  cv::Mat image(200, 200, CV_8UC3), temp;
  image.copyTo(temp);

  box   = cv::Rect(-1,-1,0,0);
  image = cv::Scalar::all(0);

  cv::namedWindow( "Box Example" );


  // Here is the crucial moment where we actually install
  // the callback. Note that we set the value of 'params' to
  // be the image we are working with so that the callback
  // will have the image to edit.
  //
  cv::setMouseCallback(
    "Box Example",
    my_mouse_callback,
    (void*)&image
  );

  // The main program loop. Here we copy the working image
  // to the temp image, and if the user is drawing, then
  // put the currently contemplated box onto that temp image.
  // Display the temp image, and wait 15ms for a keystroke,
  // then repeat.
  //
  for(;;) {

    image.copyTo(temp);
    if( drawing_box ) draw_box( temp, box );
    cv::imshow( "Box Example", temp );

    if( cv::waitKey( 15 ) == 27 ) break;
  }

  return 0;
}

// This is our mouse callback. If the user
// presses the left button, we start a box.
// When the user releases that button, then we
// add the box to the current image. When the
// mouse is dragged (with the button down) we
// resize the box.
//
void my_mouse_callback(
   int event, int x, int y, int flags, void* param
) {

  cv::Mat& image = *(cv::Mat*) param;

  switch( event ) {

    case cv::EVENT_MOUSEMOVE: {
      if( drawing_box ) {
        box.width  = x-box.x;
        box.height = y-box.y;
      }
    }
    break;

    case cv::EVENT_LBUTTONDOWN: {
      drawing_box = true;
      box = cv::Rect( x, y, 0, 0 );
    }
    break;

    case cv::EVENT_LBUTTONUP: {
      drawing_box = false;
      if( box.width < 0  ) {
        box.x += box.width;
        box.width *= -1;
      }
      if( box.height < 0 ) {
        box.y += box.height;
        box.height *= -1;
      }
      draw_box( image, box );
    }
    break;
  }

}

Sliders, trackbars, and switches

HighGUI provides a convenient slider element. In HighGUI, sliders are called trackbars. This is because their o riginal (historical) intent was for selecting a particular frame in the playback of a video. Of course, once trackbars were added to HighGUI, people began to use them for all of the usual things one might do with sliders, as well as many unusual ones (we’ll discuss some of these in “Surviving without buttons”).

As with the parent window, the slider is given a unique name (in the form of a character string) and is thereafter always referred to by that name. The HighGUI routine for creating a trackbar is:

int cv::createTrackbar(
  const string&        trackbarName,      // Handle used to identify trackbar
  const string&        windowName,        // Handle used to identify window
  int*                 value,             // Slider position gets put here
  int                  count,             // Total counts for slider at far right
  cv::TrackbarCallback onChange    = NULL,// Callback function (optional)
  void*                param       = NULL // Additional params for callback fn.
);

The first two arguments are the name for the trackbar itself and the name of the parent window to which the trackbar will be attached. When the trackbar is created, it is added to either the top or the bottom of the parent window.⁷ The trackbar will not occlude any image that is already in the window; rather, it will make the window slightly bigger. The name of the trackbar will appear as a “label” for the trackbar. As with the location of the trackbar itself, the exact location of this label depends on the operating system, but most often it is immediately to the left, as in Figure 9-2.

Figure 9-2. A simple application displaying an image; this window has two trackbars attached: Trackbar0 and Trackbar1

The next two arguments are value, a pointer to an integer that will automatically be set to the value to which the slider has been moved, and count, a numerical value for the maximum value of the slider.

The last argument is a pointer to a callback function that will be automatically called whenever the slider is moved. This is exactly analogous to the callback for mouse events. If used, the callback function must have the form specified by cv::TrackbarCallback, which means that it must match the following prototype:

void your_trackbar_callback(
  int   pos,                        // Trackbar slider position
  void* param = NULL                // Parameters from cv::setTrackbarCallback()
);

This callback is not actually required, so if you don’t want a callback, then you can simply set this value to NULL. Without a callback, the only effect of the user moving the slider will be the value of *value being updated. (Of course, if you don’t have a callback, you will be responsible for polling this value if you are going to respond to it being changed.)

The final argument to cv::createTrackbar() is params, which can be any pointer. This pointer will be passed to your callback as its params argument whenever the callback is executed. This is very helpful for, among other things, allowing you to handle trackbar events without having to introduce global variables.

Finally, here are two more routines that will allow you to programmatically read or set the value of a trackbar just by using its name:

int cv::getTrackbarPos(
  const string& trackbarName,         // Handle used to identify trackbar, label
  const string& windowName,           // Handle used to identify window
);

void cv::setTrackbarPos(
  const string& trackbarName,         // Handle used to identify trackbar, label
  const string& windowName,           // Handle used to identify window
  int   pos                           // Trackbar slider position
);

These functions allow you to read or set the value of a trackbar from anywhere in your program.

Surviving without buttons

Unfortunately, the native interface in HighGUI does not provide any explicit support for buttons. It is thus common practice, among the particularly lazy, to instead use sliders with only two positions.⁸ Another option that occurs often in the OpenCV samples in .../opencv/samples/c/ is to use keyboard shortcuts instead of buttons (see, e.g., the floodfill demo in the OpenCV source-code bundle).

Switches are just sliders (trackbars) that have only two positions, “on” (1) and “off” (0) (i.e., count has been set to 1). You can see how this is an easy way to obtain the functionality of a button using only the available trackbar tools. Depending on exactly how you want the switch to behave, you can use the trackbar callback to automatically reset the button back to 0 (as in Example 9-3; this is something like the standard behavior of most GUI “buttons”) or to automatically set other switches to 0 (which gives the effect of a “checkbox” or, with a little more work, a “radio button”).

// An example program in which the user can draw boxes on the screen.
//
#include <opencv2/opencv.hpp>
#include <iostream>

using namespace std;

//
// Using a trackbar to create a "switch" that the user can turn on and off.
// We make this value global so everyone can see it.
//
int g_switch_value = 1;
void switch_off_function() { cout << "Pause
"; }; //YOU COULD DO MORE
void switch_on_function()  { cout << "Run
"; };

// This will be the callback that we give to the trackbar.
//
void switch_callback( int position, void* ) {
  if( position == 0 ) {
    switch_off_function();
  } else {
    switch_on_function();
  }
}

void help() {
    cout << "Call: my.avi" << endl;
    cout << "Shows putting a pause button in a video." << endl;
}

int main( int argc, char** argv ) {

  cv::Mat frame; // To hold movie images
  cv::VideoCapture g_capture;
  help();
  if( argc < 2 || !g_capture.open( argv[1] ) ){
    cout << "Failed to open " << argv[1] << " video file
" << endl;
    return -1;
  }

  // Name the main window
  //
  cv::namedWindow( "Example", 1 );

  // Create the trackbar. We give it a name,
  // and tell it the name of the parent window.
  //
  cv::createTrackbar(
    "Switch",
    "Example",
    &g_switch_value,
    1,
    switch_callback
  );

  // This will cause OpenCV to idle until
  // someone hits the Esc key.
  //
  for(;;) {
    if( g_switch_value ) {
          g_capture >> frame;
          if( frame.empty() ) break;
          cv::imshow( "Example", frame);
      }
      if( cv::waitKey(10)==27 ) break;
  }

  return 0;
}

You can see that this will turn on and off just like a light switch. In our example, whenever the trackbar “switch” is set to 0, the callback executes the function switch_off_function(), and whenever it is switched on, the switch_on_function() is called.  

Getting started

If you have built your OpenCV installation with Qt support on,¹⁰ then when you open a window, it will automatically have two new features. These are the toolbar and the status bar (see Figure 9-3). These objects come up complete, with all of the contents you see in the figure. In particular, the toolbar contains buttons that allow you to pan (the first four arrow buttons), zoom (the next four buttons), save the current image (the ninth button), and pop up a properties window (more on this last one a little later).

Figure 9-3. This image is displayed with the Qt interface enabled; it shows the toolbar, the status bar, and a text overlay that, in this case, contains the name of the image

The status bar in Figure 9-3 contains information about what is under your mouse at any given moment. The x, y location is displayed, as well as the RGB value of the pixel at which the mouse currently points.

All of this you get “for free” just for compiling your code with the Qt interface enabled. If you have compiled with Qt and you don’t want these decorations, then you can simply add the CV_GUI_NORMAL flag when you call cv::namedWindow(), and they will disappear.¹¹

The actions menu

As you can see, when you create the window with CV_GUI_EXPANDED, you will see a range of buttons in the toolbar. An alternative to the toolbar, which will always be available whether you use CV_GUI_EXPANDED or not, is the pop-up menu. This pop-up menu (shown in Figure 9-4) contains the same options as the toolbar, and you can display it at any time by right-clicking on the image.

Figure 9-4. Here the Qt extended UI window is shown with the pop-up menu, which provides the same options as the toolbar (along with an explanation of the buttons and their keyboard shortcuts)

The text overlay

Another option provided by the Qt GUI is the ability to put a short-lived banner across the top of the image you are displaying. This banner is called the overlay, and appears with a shaded box around it for easy reading. This is an exceptionally handy feature if you just want to throw some simple information like the frame number or frame rate on a video, or a filename of the image you are viewing. You can display an overlay on any window, even if you are using CV_GUI_NORMAL.

int cv::displayOverlay(
  const string& name,                 // Handle used to identify window
  const string& text,                 // Text you want to display
  int           delay                 // Milliseconds to show text (0='forever')
);

The cv::displayOverlay() function takes three arguments. The first is the name of the window on which you want the overlay to appear. The second argument is whatever text you would like to appear in the window. (One word of warning here: the text has a fixed size, so if you try to cram too much stuff in there, it will just overflow.¹² By default, the text is always center justified.) The third and final argument, delay, is the amount of time (in milliseconds) that the overlay will stay in place. If delay is set to 0, then the overlay will stay in place indefinitely—or at least until you write over it with another call to cv::displayOverlay(). In general, if you call cv::displayOverlay() before the delay timer for a previous call is expired, the previous text is removed and replaced with the new text, and the timer is reset to the new delay value regardless of what is left in the timer before the new call.

Writing your own text into the status bar

In addition to the overlay, you can also write text into the status bar. By default, the status bar contains information about the pixel over which your mouse pointer is located (if any). You can see in Figure 9-3 that the status bar contains an x, y location and the RGB color value of the pixel that was under the pointer when the figure was made. You can replace this text with your own text with the cv::displayStatusBar() method:

int cv::displayStatusBar(
  const string& name,                 // Handle used to identify window
  const string& text,                 // Text you want to display
  int           delay                 // Milliseconds to show text (0='forever')
);

Unlike cv::displayOverlay(), cv::displayStatusBar() can be used only on windows that were created with the CV_GUI_EXPANDED flag (i.e., ones that have a status bar in the first place). When the delay timer is expired (if you didn’t set it to 0), then the default x, y and RGB text will reappear in the status bar.

The properties window

So far, we haven’t really discussed the last button on the toolbar (which corresponds to the last option on the pop-up menu, the one that is “darkened” in Figure 9-5). This option opens up an entirely new window called the properties window. The properties window is a convenient place to put trackbars and buttons (yes, the Qt GUI supports buttons) that you don’t want in your face all the time. It’s important to remember, however, that there is just one properties window per application, so you don’t really create it, you just configure it.

Figure 9-5. This window contains two trackbars; you can also see the control panel, which contains three push buttons, a trackbar, two radio buttons, and a checkbox

The properties window will not be available unless you have assigned some trackbars or buttons to it (more on how to do this momentarily). If it is available, then you can display it by clicking the Display Properties Window button on the toolbar (the one on the far right), clicking the identical button on the action menu, or pressing Ctrl-P while your mouse is over any window.

Trackbars revisited

In the previous section on the HighGUI native interface, we saw that it was possible to add trackbars to windows. The trackbars in Figure 9-5 were created using the same cv::createTrackbar() command we saw earlier. The only real difference is that the trackbars in Figure 9-5 are prettier than the ones we created using the non-Qt interface (recall Figure 9-2).

The important new concept in the Qt interface, however, is that we can also create trackbars in the properties window. We do so simply by creating the trackbar as we normally would, but by specifying an empty string as the window name to which the trackbar will be attached.

int contrast = 128;
cv::createTrackbar( "Contrast:", "", &contrast, 255, on_change_contrast );

For example, this fragment would create a trackbar in the properties window that would be labeled “Contrast:”, and whose value would start out at 128, with a maximum value of 255. Whenever the slider is adjusted, the callback on_change_contrast() will be called.

Creating buttons with cv::createButton()

One of the most helpful new capabilities provided by the Qt interface is the ability to create buttons. This includes normal push buttons, radio-style (mutually exclusive) buttons, and checkboxes. All buttons created are always located in the control panel.

All three styles of buttons are created with the same method:

int cv::createButton(
  const string&      buttonName,          // Handle used to identify trackbar
  cv::ButtonCallback onChange     = NULL, // Callback for button event
  void*              params,              // (Optional) params for button event
  int                buttonType   = cv::PUSH_BUTTON,  // PUSH_BUTTON or RADIOBOX
  int                initialState = 0     // Start state of the button
);

The button expects a name, buttonName, that will appear on or next to the button. If you like, you can neglect this argument and simply provide an empty string, in which case the button name will be automatically generated in a serialized manner (e.g., “button 0,” “button 1,” etc.). The second argument, onChange, is a callback that will be called whenever the button is clicked. The prototype for such a callback must match the declaration for cv::ButtonCallback, which is:

void your_button_callback(
  int   state,                            // Identifies button event type
  void* params                            // Parameters from cv::createButton()
);

When your callback is called as a result of someone clicking a button, it will be given the value state, which is derived from what just happened to the button. The pointer param that you gave to cv::createButton() will also be passed to your callback, filling its param argument.

The buttonType argument can take one of three values: cv::PUSH_BUTTON, cv::RADIOBOX, or cv::CHECKBOX. The first corresponds to your standard button—you click it, it calls your callback. In the case of the checkbox, the value will be 1 or 0 depending on whether the box was checked or unchecked. The same is true for a radio button, except that when you click a radio button, the callback is called both for the button you just clicked and for the button that is now unclicked (as a result of the mutex nature of radio buttons). All buttons in the same row—button bars, described next—are assumed to be part of the same mutex group.

When buttons are created, they are automatically organized into button bars. A button bar is a group of buttons that occupies a “row” in the properties window. Consider the following code, which generated the control panel you saw in Figure 9-5.

cv::namedWindow( "Image", cv::GUI_EXPANDED );
cv::displayOverlay( "Image", file_name, 0 );
cv::createTrackbar( "Trackbar0", "Image", &mybar0, 255 );
cv::createTrackbar( "Trackbar1", "Image", &mybar1, 255 );

cv::createButton( "", NULL, NULL, cv::PUSH_BUTTON );
cv::createButton( "", NULL, NULL, cv::PUSH_BUTTON );
cv::createButton( "", NULL, NULL, cv::PUSH_BUTTON );
cv::createTrackbar( "Trackbar2", "", &mybar1, 255 );
cv::createButton( "Button3", NULL, NULL, cv::RADIOBOX, 1 );
cv::createButton( "Button4", NULL, NULL, cv::RADIOBOX, 0 );
cv::createButton( "Button5", NULL, NULL, cv::CHECKBOX, 0 );

You will notice that Trackbar0 and Trackbar1 are created in the window called "Image", while Trackbar2 is created in an unnamed window (the properties window). The first three cv::createButton() calls are not given a name for the button, and you can see in Figure 9-5 the automatically assigned names are placed on the buttons. You will also notice in Figure 9-5 that the first three buttons are in one row, while the second group of three is on another. This is because of the trackbar.

Buttons are created one after another, each to the right of its predecessor, until (unless) a trackbar is created. Because a trackbar consumes an entire row, it is given its own row below the buttons. If more buttons are created, they will appear on a new row thereafter.¹³

Text and fonts

Just as the Qt interface allowed for much prettier trackbars and other elements, Qt also allows for much prettier and more versatile text. To write text using the Qt interface, you must first create a CvFont object,¹⁴ which you then use whenever you want to put some text on the screen. Fonts are created via the cv::fontQt() function:

CvFont fontQt(                            // Return font characterization struct
  const string& fontName,                 // e.g., "Times"
  int           pointSize,                // Size of font, using "point" system.
  cv::Scalar    color    = cv::Scalar::all(0), // BGR color as scalar (no alpha)
  int           weight   = cv::FONT_NORMAL,    // Font weight, 1-100 (Table 9-3)
  int           spacing  = 0              // Space between individual characters
);

The first argument to cv::fontQt() is the system font name. This might be something like “Times.” If your system does not have an available font with this name, then a default font will be chosen for you. The second argument, pointSize, is the size of the font (i.e., 12 = “12 point,” 14 = “14 point,” etc.) You may set this to 0, in which case a default font size (typically 12 point) will be selected for you.

The argument color can be set to any cv::Scalar and will set the color in which the font is drawn; the default value is black. weight can take one of several predefined values, or any integer between 1 and 100. The predefined aliases and their values are shown in Table 9-3.

The final argument is spacing, which controls the spacing between individual characters. It can be negative or positive.

Once you have your font, you can put text on an image (and thus on the screen)¹⁵ with cv::addText().

void cv::addText(
  cv::Mat&      image,                  // Image onto which to write
  const string& text,                   // Text to write
  cv::Point     location,               // Location of lower-left corner of text
  CvFont*       font                    // OpenCV font characerization struct
);

The arguments to cv::addText() are just what you would expect: the image to write on, the text to write, where to write it, and the font to use—with the latter being a font you defined using cv::fontQt. The location argument corresponds to the lower-left corner of the first character in text (or, more precisely, the beginning of the baseline for that character).

Setting and getting window properties

Many of the state properties of a window set at creation are queryable when you are using the Qt backend, and many of those can be changed (set) even after the window’s creation.

void   cv::setWindowProperty(
  const string& name,                   // Handle used to identify window
  int           prop_id,                // Identifies window property (Table 9-4)
  double        prop_value              // Value to which to set property
);

double cv::getWindowProperty(
  const string& name,                   // Handle used to identify window
  int           prop_id                 // Identifies window property (Table 9-4)
);

To get a window property, you need only call cv::getWindowProperty() and supply the name of the window and the property ID (prop_id argument) of the property you are interested in (see Table 9-4). Similarly, you can use cv::setWindowProperty() to set window properties with the same property IDs.

Saving and recovering window state

The Qt interface also allows the state of windows to be saved and restored. This can be very convenient, as it includes not only the location and size of your windows, but also the state of all of the trackbars and buttons. The interface state is saved with the cv::saveWindowParameters() function, which takes a single argument indicating the window to be saved:

void cv::saveWindowParameters(
  const string& name                      // Handle used to identify window
);

Once the state of the window is saved, it can be restored with the complementary cv::loadWindowParameters() function:

void cv::loadWindowParameters(
  const string& name                      // Handle used to identify window
);

The real magic here is that the load command will work correctly even if you have quit and restarted your program. The nature of how this works is not important to us here, but one detail you should know is that the state information, wherever it is saved, is saved under a key that is constructed from the executable name. So if you change the name of the executable, the state will not restore (though you can change the executable’s location without having this problem).

Interacting with OpenGL

One of the most exciting things that the Qt interface allows you to do is to generate imagery with OpenGL and overlay that imagery on top of your own image.¹⁶ This can be extremely effective for visualizing and debugging robotic or augmented-reality applications, or anywhere in which you are trying to generate a three-dimensional model from your image and want to see the result on top of the original. Figure 9-6 shows a very simple example of what you can do.

Figure 9-6. Here OpenGL is used to render a cube on top of our previous image

The basic concept is very simple: you create a callback that is an OpenGL draw function and then register it with the interface. From there, OpenCV takes care of the rest. The callback is then called every time the window is drawn (which includes whenever you call cv::imshow(), as you would with successive frames of a video stream). Your callback should match the prototype for cv::OpenGLCallback(), which means that it should be something like the following:

void your_opengl_callback(
  void* params             // (Optional) Params from cv::setOpenGLCallback()
);

Once you have your callback, you can configure the OpenGL interface with cv::set​O⁠penGLCallback():

void cv::setOpenGLCallback(
  const string&      windowName,          // Handle used to identify window
  cv::OpenGLCallback callback,            // OpenGL callback routine
  void*              params    = NULL     // (Optional) parameters for callback
);

As you can see, there is not much you need to do in order to set things up. In addition to specifying the name of the window on which the drawing will be done and supplying the callback function, you have a third argument, params, which allows you to specify a pointer that will be passed to callback whenever it is called.

In Example 9-4, we have taken a simple example from the OpenCV documentation that draws a cube in OpenGL, but we have replaced the fixed rotation angles in that cube with variables (rotx and roty), which we have made the values of the two sliders in our earlier examples. Now the user can rotate the cube with the sliders while enjoying the beautiful scenery behind it.

void on_opengl( void* param ) {

  glMatrixModel( GL_MODELVIEW );
  glLoadIdentity();

  glTranslated( 0.0, 0.0, -1.0 );

  glRotatef( rotx, 1, 0, 0 );
  glRotatef( roty, 0, 1, 0 );
  glRotatef( 0, 0, 0, 1 );

  static const int coords[6][4][3] = {
    { { +1, -1, -1 }, { -1, -1, -1 }, { -1, +1, -1 }, { +1, +1, -1 } },
    { { +1, +1, -1 }, { -1, +1, -1 }, { -1, +1, +1 }, { +1, +1, +1 } },
    { { +1, -1, +1 }, { +1, -1, -1 }, { +1, +1, -1 }, { +1, +1, +1 } },
    { { -1, -1, -1 }, { -1, -1, +1 }, { -1, +1, +1 }, { -1, +1, -1 } },
    { { +1, -1, +1 }, { -1, -1, +1 }, { -1, -1, -1 }, { +1, -1, -1 } },
    { { -1, -1, +1 }, { +1, -1, +1 }, { +1, +1, +1 }, { -1, +1, +1 } }
  };

  for (int i = 0; i < 6; ++i) {
    glColor3ub( i*20, 100+i*10, i*42 );
    glBegin(GL_QUADS);
    for (int j = 0; j < 4; ++j) {
      glVertex3d(
        0.2 * coords[i][j][0],
        0.2 * coords[i][j][1],
        0.2 * coords[i][j][2]
      );
    }
    glEnd();
  }
}

An example of OpenCV and Qt

Here we will show an example of using the actual Qt toolkit to write a program that reads a video file and displays it on the screen. There are several subtleties, some of which have to do with how one uses Qt, and others to do with OpenCV. Of course, we will focus on the latter, but it is worth taking a moment to notice how the former affects our current goal.

Example 9-5 shows the top-level code for our program; it just creates a Qt application and adds our QMoviePlayer widget. Everything interesting will happen inside that object.

#include <QApplication>
#include <QLabel>
#include <QMoviePlayer.hpp>
int main( int argc, char* argv[] ) {

  QApplication app( argc, argv );

  QMoviePlayer mp;
  mp.open( argv[1] );
  mp.show();

  return app.exec();
}

The interesting stuff is in the QMoviePlayer object. Let’s take a look at the header file which defines that object in Example 9-6.

#include "ui_QMoviePlayer.h"
#include <opencv2/opencv.hpp>
#include <string>

using namespace std;

class QMoviePlayer : public QWidget {

  Q_OBJECT;

  public:
  QMoviePlayer( QWidget *parent = NULL );
  virtual ~QMoviePlayer() {;}

  bool open( string file );

  private:
  Ui::QMoviePlayer  ui;
  cv::VideoCapture m_cap;

  QImage  m_qt_img;
  cv::Mat m_cv_img;
  QTimer* m_timer;

  void paintEvent( QPaintEvent* q );
  void _copyImage( void );

  public slots:
  void nextFrame();

};

There is a lot going on here. The first thing that happens is the inclusion of the file ui_QMoviePlayer.h. This file was automatically generated by the Qt Designer. What matters here is that it is just a QWidget that contains nothing but a QFrame called frame. The member ui::QMoviePlayer is that interface object that is defined in ui_QMoviePlayer.h.

In this file, there is also a QImage called m_qt_img and a cv::Mat called m_cv_img. These will contain the Qt and OpenCV representations of the image we are getting from our video. Finally, there is a QTimer, which is what will take the place of cv::waitKey(), allowing us to replay the video frames at the correct rate. The remaining functions will become clear as we look at their actual definitions in QMoviePlayer.cpp (Example 9-7).

#include "QMoviePlayer.hpp"
#include <QTimer>
#include <QPainter>

QMoviePlayer::QMoviePlayer( QWidget *parent )
 : QWidget( parent )
{
  ui.setupUi( this );
}

The top-level constructor for QMoviePlayer just calls the setup function, which was automatically built for us for the UI member.

bool QMoviePlayer::open( string file ) {

  if( !m_cap.open( file ) ) return false;

  // If we opened the file, set up everything now:
  //
  m_cap.read( m_cv_img );
  m_qt_img = QImage(
    QSize( m_cv_img.cols, m_cv_img.rows ),
    QImage::Format_RGB888
  );
  ui.frame->setMinimumSize( m_qt_img.width(), m_qt_img.height() );
  ui.frame->setMaximumSize( m_qt_img.width(), m_qt_img.height() );
  _copyImage();

  m_timer = new QTimer( this );
  connect(
    m_timer,
    SIGNAL( timeout() ),
    this,
    SLOT( nextFrame() )
  );
  m_timer->start( 1000. / m_cap.get( cv::CAP_PROP_FPS ) );

  return true;

}

When an open() call is made on the QMoviePlayer, several things have to happen. The first is that the cv::VideoCapture member object m_cap needs to be opened. If that fails, we just return. Next we read the first frame into our OpenCV image member m_cv_img. Once we have this, we can set up the Qt image object m_qt_img, giving it the same size as the OpenCV image. After that, we resize the frame object in the UI element to be the same size as the incoming images as well.

We will look at the call to QMoviePlayer::_copyImage() in a moment; this is going to handle the very important process of converting the image we have already captured into m_cv_img onto the Qt image m_qt_img, which we are actually going to have Qt paint onto the screen for us.

The last thing we do in QMoviePlayer::open() is set up a QTimer such that, when it “goes off,” it will call the function QMoviePlayer::nextFrame() (which will, not surprisingly, get the next frame). The call to m_timer->start() is how we both start the timer running and indicate that it should go off at the correct rate implied by cv::CAP_PROP_FPS (i.e., 1,000 milliseconds divided by the frame rate).

void QMoviePlayer::_copyImage( void ) {

  // Copy the image data into the Qt QImage
  //
  cv::Mat cv_header_to_qt_image(
    cv::Size(
      m_qt_img.width(),
      m_qt_img.height()
    ),
    CV_8UC3,
    m_qt_img.bits()
  );
  cv::cvtColor( m_cv_img, cv_header_to_qt_image, cv::BGR2RGB );

}

The QMoviePlayer::_copyImage() function is responsible for copying the image from the buffer m_cv_img into the Qt image buffer m_qt_img. The way we do this shows off a nice feature of the cv::Mat object. First, we define a cv::Mat object called cv_header_to_qt_image. When we define that object, we actually tell it what area to use for its data area, and hand it the data area for the Qt QImage object m_qt_img.bits(). We then call cv::cvtColor to do the copying, which handles the subtlety that OpenCV prefers BGR ordering, while Qt prefers RGB.

void QMoviePlayer::nextFrame() {

  // Nothing to do if capture object is not open
  //
  if( !m_cap.isOpened() ) return;

  m_cap.read( m_cv_img );
  _copyImage();

  this->update();

}

The QMoviePlayer::nextFrame() function actually handles the reading of subsequent frames. Recall that this routine is called whenever the QTimer expires. It reads the new image into the OpenCV buffer, calls QMoviePlayer::_copyImage() to copy it into the Qt buffer, and then makes an update call on the QWidget that this is all part of (so that Qt knows that something has changed).

void QMoviePlayer::paintEvent( QPaintEvent* e ) {

  QPainter painter( this );

  painter.drawImage( QPoint( ui.frame->x(), ui.frame->y()), m_qt_img );

}

Last but not least is the QMoviePlayer::paintEvent() function. This is called by Qt whenever it is necessary to actually draw the QMoviePlayer widget. This function just creates a QPainter and tells it to draw the current Qt image m_qt_img (starting at the corner of the screen).

An example of OpenCV and wxWidgets

In Example 9-8, we will use a different cross-platform toolkit, wxWidgets. The wxWidgets toolkit is similar in many ways to Qt in terms of its GUI components but, naturally, it is in the details that things tend to become difficult. As with the Qt example, we will have one top-level file that basically puts everything in place and a code and header file pair that defines an object that encapsulates our example task of playing a video. This time our object will be called WxMoviePlayer and we will build it based on the UI classes provided by wxWidgets.

#include "wx/wx.h"
#include "WxMoviePlayer.hpp"

// Application class, the top level object in wxWidgets
//
class MyApp : public wxApp {
  public:
    virtual bool OnInit();
};

// Behind the scenes stuff to create a main() function and attach MyApp
//
DECLARE_APP( MyApp );
IMPLEMENT_APP( MyApp );

// When MyApp is initialized, do these things.
//
bool MyApp::OnInit() {

  wxFrame* frame = new wxFrame( NULL, wxID_ANY, wxT("ch4_wx") );
  frame->Show( true );

  WxMoviePlayer* mp = new WxMoviePlayer(
    frame,
    wxPoint( -1, -1 ),
    wxSize( 640, 480 )
  );
  mp->open( wxString(argv[1]) );
  mp->Show( true );

  return true;

}

The structure here looks a little more complicated than the Qt example, but the content is very similar. The first thing we do is create a class definition for our application, which we derive from the library class wxApp. The only thing different about our class is that it will overload the MyApp::OnInit() function with our own content. After declaring class MyApp, we call two macros: DECLARE_APP() and IMPLEMENT_APP(). In short, these are creating the main() function and installing an instance of MyApp as “the application.” The last thing we do in our main program is to actually fill out the function MyApp::OnInit() that will be called when our program starts. When MyApp::OnInit() is called, it creates the window (called a frame in wxWidgets), and an instance of our WxMoviePlayer object in that frame. It then calls the open method on the WxMoviePlayer and hands it the name of the movie file we want to open.

Of course, all of the interesting stuff is happening inside of the WxMoviePlayer object. Example 9-9 shows the header file for that object.

#include "opencv2/opencv.hpp"

#include "wx/wx.h"
#include <string>

#define TIMER_ID 0

using namespace std;

class WxMoviePlayer : public wxWindow {

  public:
    WxMoviePlayer(
      wxWindow*      parent,
      const wxPoint& pos,
      const wxSize&  size
    );
    virtual ~WxMoviePlayer() {};
    bool open( wxString file );

  private:

    cv::VideoCapture m_cap;
    cv::Mat          m_cv_img;
    wxImage          m_wx_img;
    wxBitmap         m_wx_bmp;
    wxTimer*         m_timer;
    wxWindow*        m_parent;

    void _copyImage( void );

    void OnPaint( wxPaintEvent& e );
    void OnTimer( wxTimerEvent& e );
    void OnKey(   wxKeyEvent&   e );

  protected:
    DECLARE_EVENT_TABLE();
};

There are several important things to notice in this declaration. The WxMoviePlayer object is derived from wxWindow, which is the generic class used by wxWidgets for just about anything that will be visible on the screen. We have three event-handling methods, OnPaint(), onTimer(), and OnKey(); these will handle drawing, getting a new image from the video, and closing the file with the Esc key, respectively. Finally, you will notice that there is an object of type wxImage and an object of type wxBitmap, in addition to the OpenCV cv:Mat type image. In wxWidgets, bitmaps (which are operating system dependent) are distinguished from images (which are device-independent representations of image data). The exact role of these two will be clear shortly as we look at the code file WxMoviePlayer.cpp (see Example 9-10).

#include "WxMoviePlayer.hpp"

BEGIN_EVENT_TABLE( WxMoviePlayer, wxWindow )
  EVT_PAINT( WxMoviePlayer::OnPaint )
  EVT_TIMER( TIMER_ID, WxMoviePlayer::OnTimer )
  EVT_CHAR( WxMoviePlayer::OnKey )
END_EVENT_TABLE()

The first thing we do is to set up the callbacks that will be associated with individual events. We do this through macros provided by the wxWidgets framework.¹⁷

WxMoviePlayer::WxMoviePlayer(
  wxWindow*      parent,
  const wxPoint& pos,
  const wxSize&  size
) : wxWindow( parent, -1, pos, size, wxSIMPLE_BORDER ) {
  m_timer         = NULL;
  m_parent        = parent;
}

When the movie player is created, its timer element is NULL (we will set that up when we actually have a video open). We do take note of the parent of the player, however. (In this case, that parent will be the wxFrame we created to put it in.) We will need to know which frame is the parent when it comes time to close the application in response to the Esc key.

void WxMoviePlayer::OnPaint( wxPaintEvent& event ) {
  wxPaintDC dc( this );

  if( !dc.Ok() ) return;

  int x,y,w,h;
  dc.BeginDrawing();
    dc.GetClippingBox( &x, &y, &w, &h );
    dc.DrawBitmap( m_wx_bmp, x, y );
  dc.EndDrawing();

  return;
}

The WxMoviePlayer::OnPaint() routine is called whenever the window needs to be repainted on screen. Notice that when we execute WxMoviePlayer::OnPaint(), the information we need to actually do the painting is assumed to be in m_wx_bpm, the wxBitmap object. Because the wxBitmap is the system-dependent representation, it is already prepared to be copied to the screen. The next two methods, WxMoviePlayer::_copyImage() and WxMoviePlayer::open(), will show how it got created in the first place.

void WxMoviePlayer::_copyImage( void ) {

  m_wx_bmp = wxBitmap( m_wx_img );

  Refresh( FALSE ); // indicate that the object is dirty
  Update();

}

The WxMoviePlayer::_copyImage() method will get called whenever a new image is read from the cv::VideoCapture object. It doesn’t appear to do much, but actually a lot is going on in its short body. First and foremost is the construction of the wxBitmap m_wx_bmp from the wxImage m_wx_img. The constructor is handling the conversion from the abstract representation used by wxImage (which, we will see, looks very much like the representation used by OpenCV) to the device- and system-specific representation used by your particular machine. Once that copy is done, a call to Refresh() indicates that the widget is “dirty” and needs redrawing, and the subsequent call to Update() indicates that the time for that redrawing is now.

bool WxMoviePlayer::open( wxString file ) {

  if( !m_cap.open( std::string( file.mb_str() ) )) {
    return false;
  }

  // If we opened the file, set up everything now:
  //
  m_cap.read( m_cv_img );

  m_wx_img = wxImage(
    m_cv_img.cols,
    m_cv_img.rows,
    m_cv_img.data,
    TRUE  // static data, do not free on delete()
  );

  _copyImage();

  m_timer = new wxTimer( this, TIMER_ID );
  m_timer->Start( 1000. / m_cap.get( cv::CAP_PROP_FPS ) );

  return true;

}

The WxMoviePlayer::open() method also does several important things. The first is to actually open the cv::VideoCapture object, but there is a lot more to be done. Next, an image is read off of the player and is used to create a wxImage object that “points at” the OpenCV cv::Mat image. This is the opposite philosophy to the one we used in the Qt example: in this case, it turns out to be a little more convenient to create the cv::Mat first and have it own the data, and then to create the GUI toolkit’s image object second and have it be just a header to that existing data. Next, we call WxMoviePlayer::_copyImage(), and that function converts the OpenCV image m_cv_img into the native bitmap for us.

Finally, we create a wxTimer object and tell it to wake us up every few milliseconds—with that number being computed from the FPS reported by the cv::VideoCapture object. Whenever that timer expires, a wxTimerEvent is generated and passed to WxMoviePlayer::OnTimer(), which you will recall is the handler of such events.

void WxMoviePlayer::OnTimer( wxTimerEvent& event ) {

  if( !m_cap.isOpened() ) return;

  m_cap.read( m_cv_img );
  cv::cvtColor( m_cv_img, m_cv_img, cv::BGR2RGB );
  _copyImage();

}

That handler doesn’t do too much; primarily it just reads a new frame from the video, converts that frame from BGR to RGB for display, and then calls our WxMoviePlayer::_copyImage(), which makes the next bitmap for us.

void WxMoviePlayer::OnKey( wxKeyEvent& e ) {

  if( e.GetKeyCode() == WXK_ESCAPE ) m_parent->Close();

}

Finally, we have our handler for any keypresses. It simply checks to see if that key was the Esc key, and if so, closes the program. Note that we do not close the WxMoviePlayer object, but rather the parent frame. Closing the frame is the same as closing the window any other way; it shuts down the application.

An example of OpenCV and the Windows Template Library

In this example, we will use the native windows GUI API.¹⁸ The Windows Template Library (WTL) is a very thin C++ wrapper around the raw Win32 API. WTL applications are structured similarly to MFC, in that there is an application/document-view structure. For the purposes of this sample, we will start by running the WTL Application Wizard from within Visual Studio (Figure 9-7), creating a new SDI Application, and ensuring that Use a View Window is selected under User Interface Features (it should be, by default).

Figure 9-7. The WTL Application Wizard

The exact filenames generated by the wizard will depend on the name you give your project. For Example 9-11, the project is named OpenCVTest, and we will mostly be working in the COpenCVTestView class.

class COpenCVTestView : public CWindowImpl<COpenCVTestView> {

public:
  DECLARE_WND_CLASS(NULL)

  bool OpenFile(std::string file);
  void _copyImage();


  BOOL PreTranslateMessage(MSG* pMsg);

  BEGIN_MSG_MAP(COpenCVTestView)
    MESSAGE_HANDLER(WM_ERASEBKGND, OnEraseBkgnd)
    MESSAGE_HANDLER(WM_PAINT, OnPaint)
    MESSAGE_HANDLER(WM_TIMER, OnTimer)
  END_MSG_MAP()

// Handler prototypes (uncomment arguments if needed):
//  LRESULT MessageHandler(
//    UINT    /*uMsg*/,
//    WPARAM  /*wParam*/,
//    LPARAM  /*lParam*/,
//    BOOL&   /*bHandled*/
//  );
//  LRESULT CommandHandler(
//    WORD    /*wNotifyCode*/,
//    WORD    /*wID*/,
//    HWND    /*hWndCtl*/,
//    BOOL&   /*bHandled*/
//  );
//  LRESULT NotifyHandler(
//    int     /*idCtrl*/,
//    LPNMHDR /*pnmh*/,
//    BOOL&   /*bHandled*/
//  );

  LRESULT OnPaint(
    UINT    /*uMsg*/,
    WPARAM  /*wParam*/,
    LPARAM  /*lParam*/,
    BOOL&   /*bHandled*/
  );
  LRESULT OnTimer(
    UINT    /*uMsg*/,
    WPARAM  /*wParam*/,
    LPARAM  /*lParam*/,
    BOOL&   /*bHandled*/
  );
  LRESULT OnEraseBkgnd(
    UINT    /*uMsg*/,
    WPARAM  /*wParam*/,
    LPARAM  /*lParam*/,
    BOOL&   /*bHandled*/
  );

private:
  cv::VideoCapture m_cap;
  cv::Mat          m_cv_img;

  RGBTRIPLE*       m_bitmapBits;
};

The structure here is very similar to the preceding wxWidgets example. The only change outside of the view code is for the Open menu item handler, which will be in your CMainFrame class. It will need to call into the view class to open the video:

LRESULT CMainFrame::OnFileOpen(
  WORD /*wNotifyCode*/,
  WORD /*wID*/,
  HWND /*hWndCtl*/,
  BOOL& /*bHandled*/
) {
  WTL::CFileDialog dlg(TRUE);
  if (IDOK == dlg.DoModal(m_hWnd)) {
    m_view.OpenFile(dlg.m_szFileName);
  }
  return 0;
}

bool COpenCVTestView::OpenFile(std::string file) {

  if( !m_cap.open( file ) ) return false;

  // If we opened the file, set up everything now:
  //
  m_cap.read( m_cv_img );

  // could create a DIBSection here, but let's just allocate memory for raw bits
  //
  m_bitmapBits = new RGBTRIPLE[m_cv_img.cols * m_cv_img.rows];

  _copyImage();

  SetTimer(0, 1000.0f / m_cap.get( cv::CAP_PROP_FPS ) );

  return true;
}


void COpenCVTestView::_copyImage() {

  // Copy the image data into the bitmap
  //
  cv::Mat cv_header_to_qt_image(
    cv::Size(
      m_cv_img.cols,
      m_cv_img.rows
    ),
    CV_8UC3,
    m_bitmapBits
  );
  cv::cvtColor( m_cv_img, cv_header_to_qt_image, cv::BGR2RGB );
}

LRESULT COpenCVTestView::OnPaint(
  UINT   /* uMsg     */,
  WPARAM /* wParam   */,
  LPARAM /* lParam   */,
  BOOL&  /* bHandled */
) {
  CPaintDC dc(m_hWnd);

  WTL::CRect rect;
  GetClientRect(&rect);

  if( m_cap.isOpened() ) {

    BITMAPINFO bmi = {0};
    bmi.bmiHeader.biSize = sizeof(bmi.bmiHeader);
    bmi.bmiHeader.biCompression = BI_RGB;
    bmi.bmiHeader.biWidth       = m_cv_img.cols;

    // note that bitmaps default to bottom-up, use negative height to
    // represent top-down
    //
    bmi.bmiHeader.biHeight = m_cv_img.rows * -1;

    bmi.bmiHeader.biPlanes = 1;
    bmi.bmiHeader.biBitCount = 24;  // 32 if you use RGBQUADs for the bits

    dc.StretchDIBits(
      0,                     0,
      rect.Width(),          rect.Height(),
      0,                     0,
      bmi.bmiHeader.biWidth, abs(bmi.bmiHeader.biHeight),
      m_bitmapBits,
      &bmi,
      DIB_RGB_COLORS,
      SRCCOPY
    );

  } else {

    dc.FillRect(rect, COLOR_WINDOW);

  }

  return 0;
}

LRESULT COpenCVTestView::OnTimer(
  UINT   /* uMsg     */,
  WPARAM /* wParam   */,
  LPARAM /* lParam   */,
  BOOL&  /* bHandled */
) {
  // Nothing to do if capture object is not open
  //
  if( !m_cap.isOpened() ) return 0;

  m_cap.read( m_cv_img );
  _copyImage();

  Invalidate();

  return 0;
}

LRESULT COpenCVTestView::OnEraseBkgnd(
  UINT   /* uMsg     */,
  WPARAM /* wParam   */,
  LPARAM /* lParam   */,
  BOOL&  /* bHandled */
) {
  // since we completely paint our window in the OnPaint handler, use
  // an empty background handler
  return 0;
}

This code illustrates how to use bitmap-based drawing in a C++ application in Windows. This method is simpler but less efficient than using DirectShow to handle the video stream.