Chapter 6. Drawing and Annotating

Drawing Things

We often want to draw some kind of picture, or to draw something on top of an image obtained from somewhere else. Toward this end, OpenCV provides a menagerie of functions that will allow us to make lines, squares, circles, and the like.

OpenCV’s drawing functions work with images of any depth, but most of them affect only the first three channels—defaulting to only the first channel in the case of single-channel images. Most of the drawing functions support a color, a thickness, a line type (which really refers to whether to anti-alias lines), and subpixel alignment of objects.

When you specify colors, the convention is to use the cv::Scalar object, even though only the first three values are used most of the time. (It is sometimes convenient to be able to use the fourth value in a cv::Scalar to represent an alpha channel, but the drawing functions do not currently support alpha blending.) Also, by convention, OpenCV uses BGR ordering1 for converting multichannel images to color renderings (this is what is used by the draw function imshow(), which actually paints images onto your screen for viewing). Of course, you don’t have to use this convention, and it might not be ideal if you are using data from some other library with OpenCV headers on top of it. In any case, the core functions of the library are always agnostic to any “meaning” you might assign to a channel.

Line Art and Filled Polygons

Functions that draw lines of one kind or another (segments, circles, rectangles, etc.) will usually accept a thickness and lineType parameter. Both are integers, but the only accepted values for the latter are 4, 8, or cv::LINE_AA. thickness is the thickness of the line measured in pixels. For circles, rectangles, and all of the other closed shapes, the thickness argument can also be set to cv::FILLED (which is an alias for −1).  In that  case, the result is that the drawn figure will be filled in the same color as the edges. The lineType argument indicates whether the lines should be “4-connected,” “8-connected,” or anti-aliased. For the first two examples in Figure 6-1, the Bresenham algorithm  is used, while for the anti-aliased lines, Gaussian filtering is used. Wide lines are always drawn with rounded ends.

The same line as it would be rendered using the 4-connected (a), 8-connected (b), and anti-aliased (c) line types
Figure 6-1. The same line as it would be rendered using the 4-connected (a), 8-connected (b), and anti-aliased (c) line types

For the drawing algorithms listed in Table 6-1, endpoints (lines), center points (circles), corners (rectangles), and so on are typically specified as integers. However, these algorithms support subpixel alignment through the shift argument. Where shift is available, it is interpreted as the number of bits in the integer arguments to treat as fractional bits. For example, if you say you want a circle centered at (5, 5), but set shift to 1, then the circle will be drawn at (2.5, 2.5). The effect of this will typically be quite subtle, and depend on the line type used. The effect is most noticeable for anti-aliased lines.

Table 6-1. Drawing functions
Function Description
cv::circle() Draw a simple circle
cv::clipLine() Determine if a line is inside a given box
cv::ellipse() Draw an ellipse, which may be tilted or an elliptical arc
cv::ellipse2Poly() Compute a polygon approximation to an elliptical arc
cv::fillConvexPoly() Draw filled versions of simple polygons
cv::fillPoly() Draw filled versions of arbitrary polygons
cv::line() Draw a simple line
cv::rectangle() Draw a simple rectangle
cv::polyLines() Draw multiple polygonal curves

The following sections describe the details of each function in Table 6-1.

cv::circle()

void circle(
  cv::Mat&          img,                  // Image to be drawn on
  cv::Point         center,               // Location of circle center
  int               radius,               // Radius of circle
  const cv::Scalar& color,                // Color, RGB form
  int               thickness = 1,        // Thickness of line
  int               lineType  = 8,        // Connectedness, 4 or 8
  int               shift     = 0         // Bits of radius to treat as fraction
);

The first argument to cv::circle() is just your image, img. Next are the center, a two-dimensional point, and the radius. The remaining arguments are the standard color, thickness, lineType, and shift. The shift is applied to both the radius and the center location.

cv::clipLine()

bool clipLine(                          // True if any part of line in 'imgRect'
  cv::Rect          imgRect,            // Rectangle to clip to
  cv::Point&        pt1,                // First endpoint of line, overwritten
  cv::Point&        pt2                 // Second endpoint of line, overwritten
);

bool clipLine(                          // True if any part of line in image size
  cv::Size          imgSize,            // Size of image, implies rectangle at 0,0
  cv::Point&        pt1,                // First endpoint of line, overwritten
  cv::Point&        pt2                 // Second endpoint of line, overwritten
);

This function is used to determine if a line specified by the two points pt1 and pt2 lies inside a rectangular boundary. In the first version, a cv::Rect is supplied and the line is compared to that rectangle. cv::clipLine() will return False only if the line is entirely outside of the specified rectangular region. The second version is the same except it takes a cv::Size argument. Calling this second version is equivalent to calling the first version with a rectangle whose (x, y) location is (0, 0).

cv::ellipse()

bool ellipse(
  cv::Mat&               img,               // Image to be drawn on
  cv::Point              center,            // Location of ellipse center
  cv::Size               axes,              // Length of major and minor axes
  double                 angle,             // Tilt angle of major axis
  double                 startAngle,        // Start angle for arc drawing
  double                 endAngle,          // End angle for arc drawing
  const cv::Scalar&      color,             // Color, BGR form
  int                    thickness = 1,     // Thickness of line
  int                    lineType  = 8,     // Connectedness, 4 or 8
  int                    shift     = 0      // Bits of radius to treat as fraction
);

bool ellipse(
  cv::Mat&               img,               // Image to be drawn on
  const cv::RotatedRect& rect,              // Rotated rectangle bounds ellipse
  const cv::Scalar&      color,             // Color, BGR form
  int                    thickness = 1,     // Thickness of line
  int                    lineType  = 8,     // Connectedness, 4 or 8
  int                    shift     = 0      // Bits of radius to treat as fraction
);

The cv::ellipse() function is very similar to the cv::circle() function, with the primary difference being the axes argument, which is of type cv::Size. In this case, the height and width arguments represent the length of the ellipse’s major and minor axes. The angle is the angle (in degrees) of the major axis, which is measured counterclockwise from horizontal (i.e., from the x-axis). Similarly, the startAngle and endAngle indicate (also in degrees) the angle for the arc to start and for it to finish. Thus, for a complete ellipse, you must set these values to 0 and 360, respectively.

The alternate way to specify the drawing of an ellipse is to use a bounding box. In this case, the argument box of type cv::RotatedRect completely specifies both the size and the orientation of the ellipse. Both methods of specifying an ellipse are illustrated in Figure 6-2.

An elliptical arc specified by the major and minor axes with tilt angle (left); a similar ellipse specified using a cv::RotatedRect (right)
Figure 6-2. An elliptical arc specified by the major and minor axes with tilt angle (left); a similar ellipse specified using a cv::RotatedRect (right)

cv::ellipse2Poly()

void ellipse2Poly(
  cv::Point              center,          // Location of ellipse center
  cv::Size               axes,            // Length of major and minor axes
  double                 angle,           // Tilt angle of major axis
  double                 startAngle,      // Start angle for arc drawing
  double                 endAngle,        // End angle for arc drawing
  int                    delta,           // Angle between sequential vertices
  vector<cv::Point>&     pts              // Result, STL-vector of points
);

The cv::ellipse2Poly() function is used internally by cv::ellipse() to compute elliptical arcs, but you can call it yourself as well. Given information about an elliptical arc (center, axes, angle, startAngle, and endAngle—all as defined in cv::ellipse()) and a parameter delta, which specifies the angle between subsequent points you want to sample, cv::ellipse2Poly() computes a sequence of points that form a polygonal approximation to the elliptical arc you specified. The computed points are returned in the vector<> pts.

cv::fillConvexPoly()

void fillConvexPoly(
  cv::Mat&          img,                  // Image to be drawn on
  const cv::Point*  pts,                  // C-style array of points
  int               npts,                 // Number of points in 'pts'
  const cv::Scalar& color,                // Color, BGR form
  int               lineType  = 8,        // Connectedness, 4 or 8
  int               shift     = 0         // Bits of radius to treat as fraction
);

This function draws a filled polygon. It is much faster than cv::fillPoly() (described next) because it uses a much simpler algorithm. The algorithm used by cv::fillConvexPoly(), however, will not work correctly if the polygon you pass to it has self-intersections.2 The points in pts are treated as sequential, and a segment from the last point in pts and the first point is implied (i.e., the polygon is assumed to be closed).

cv::fillPoly()

void fillPoly(
  cv::Mat&          img,                  // Image to be drawn on
  const cv::Point*  pts,                  // C-style array of arrays of points
  int               npts,                 // Number of points in 'pts[i]'
  int               ncontours,            // Number of arrays in 'pts'
  const cv::Scalar& color,                // Color, BGR form
  int               lineType = 8,         // Connectedness, 4 or 8
  int               shift    = 0,         // Bits of radius to treat as fraction
  cv::Point         offset   = Point()    // Uniform offset applied to all points
);

This function draws any number of filled polygons. Unlike cv::fillConvexPoly(), it can handle polygons with self-intersections. The ncontours argument specifies how many different polygon contours there will be, and the npts argument is a C-style array that indicates how many points there are in each contour (i.e., npts[i] indicates how many points there are in polygon i). pts is a C-style array of C-style arrays containing all of the points in those polygons (i.e., pts[i][j] contains the jth point in the ith polygon). cv::fillPoly() also has one additional argument, offset, which is a pixel offset that will be applied to all vertex locations when the polygons are drawn. The polygons are assumed to be closed (i.e., a segment from the last element of pts[i][] to the first element will be assumed).

cv::line()

void line(
  cv::Mat&          img,                  // Image to be drawn on
  cv::Point         pt1,                  // First endpoint of line
  cv::Point         pt2                   // Second endpoint of line
  const cv::Scalar& color,                // Color, BGR form
  int               lineType = 8,         // Connectedness, 4 or 8
  int               shift    = 0          // Bits of radius to treat as fraction
);

The function cv::line() draws a straight line from pt1 to pt2 in the image img. Lines are automatically clipped by the image boundaries. 

cv::rectangle()

void rectangle(
  cv::Mat&          img,                  // Image to be drawn on
  cv::Point         pt1,                  // First corner of rectangle
  cv::Point         pt2                   // Opposite corner of rectangle
  const cv::Scalar& color,                // Color, BGR form
  int               lineType = 8,         // Connectedness, 4 or 8
  int               shift    = 0          // Bits of radius to treat as fraction
);

void rectangle(
  cv::Mat&          img,                  // Image to be drawn on
  cv::Rect          r,                    // Rectangle to draw
  const cv::Scalar& color,                // Color, BGR form
  int               lineType = 8,         // Connectedness, 4 or 8
  int               shift    = 0          // Bits of radius to treat as fraction
);

The cv::rectangle() function draws a rectangle with corners pt1 to pt2 in the image img. An alternate form of this function allows the rectangle’s location and size to be specified by a single cv::Rect argument, r.

cv::polyLines()

void polyLines(
  cv::Mat&          img,                  // Image to be drawn on
  const cv::Point*  pts,                  // C-style array of arrays of points
  int               npts,                 // Number of points in 'pts[i]'
  int               ncontours,            // Number of arrays in 'pts'
  bool              isClosed,             // If true, connect last and first pts
  const cv::Scalar& color,                // Color, BGR form
  int               lineType = 8,         // Connectedness, 4 or 8
  int               shift    = 0          // Bits of radius to treat as fraction
);

This function draws any number of unfilled polygons. It can handle general polygons, including polygons with self-intersections. The ncontours argument specifies how many different polygon contours there will be, and the npts argument is a C-style array that indicates how many points there are in each contour (i.e., npts[i] indicates how many points there are in polygon i). pts is a C-style array of C-style arrays containing all of the points in those polygons (i.e., pts[i][j] contains the jth point in the ith polygon). Polygons are not assumed to be closed. If the argument isClosed is true, then a segment from the last element of pts[i][] to the first element will be assumed. Otherwise, the contour is taken to be an open contour containing only npts[i]-1 segments between the npts[i] points listed. 

cv::LineIterator

LineIterator::LineIterator(
  cv::Mat&       img,                  // Image to be drawn on
  cv::Point      pt1,                  // First endpoint of line
  cv::Point      pt2                   // Second endpoint of line
  int            lineType = 8,         // Connectedness, 4 or 8
  bool           leftToRight = false   // If true, always start steps on the left
);

The cv::LineIterator object is an iterator that is used to get each pixel of a raster line in sequence. The line iterator is our first example of a functor in OpenCV. We will see several more of these “objects that do stuff” in the next chapter. The constructor for the line iterator takes the two endpoints for the line as well as a line-type specifier and an additional Boolean that indicates which direction the line should be traversed.

Once initialized, the number of pixels in the line is stored in the member integer cv::LineIterator::count. The overloaded  dereferencing operator cv::LineIterator::operator*() returns  a pointer of type uchar*, which points to the “current” pixel. The current pixel starts at one end of the line and is incremented by means of the overloaded increment operator cv::LineIterator::operator++(). The actual traversal is done according to the Bresenham algorithm mentioned earlier.

The purpose of the cv::LineIterator is to make it possible for you to take some specific action on each pixel along the line. This is particularly handy when you are creating special effects such as switching the color of a pixel from black to white and white to black (i.e., an XOR operation on a binary image).

When accessing an individual “pixel,” remember that this pixel may have one or many channels and it might be any kind of image depth. The return value from the dereferencing operator is always uchar*, so you are responsible for casting that pointer to the correct type. For example, if your image were a three-channel image of 32-bit floating-point numbers and your iterator were called iter, then you would want to cast the return (pointer) value of the dereferencing operator like this: (Vec3f*)*iter.

Note

The style of the overloaded dereferencing operator cv::LineIterator::operator*() is slightly different  than what you are probably used to from libraries like STL. The difference is that the return value from the iterator is itself a pointer, so the iterator behaves not like a pointer, but like a pointer to a pointer.

Fonts and Text

One additional form of drawing is to draw text. Of course, text creates its own set of complexities, but—as always—OpenCV is more concerned with providing a simple “down and dirty” solution that will work for simple cases than a robust, complex solution (which would be redundant anyway given the capabilities of other libraries). Table 6-2 lists OpenCV’s two text drawing functions.

Table 6-2. Text drawing functions
Function Description
cv::putText() Draw the specified text in an image
cv::getTextSize() Determine the width and height of a text string

cv::putText()

void cv::putText(
  cv::Mat&      img,                      // Image to be drawn on
  const string& text,                     // write this (often from cv::format)
  cv::Point     origin,                   // Upper-left corner of text box
  int           fontFace,                 // Font (e.g., cv::FONT_HERSHEY_PLAIN)
  double        fontScale,                // size (a multiplier, not "points"!)
  cv::Scalar    color,                    // Color, RGB form
  int           thickness = 1,            // Thickness of line
  int           lineType  = 8,            // Connectedness, 4 or 8
  bool          bottomLeftOrigin = false  // true='origin at lower left'
);

This function is OpenCV’s one main text drawing routine; it simply throws some text onto an image. The text indicated by text is printed with its upper-left corner of the text box at origin and in the color indicated by color, unless the bottomLeftOrigin flag is true, in which case the lower-left corner of the text box is located at origin. The font used is selected by the fontFace argument, which can be any of those listed in Table 6-3.

Table 6-3. Available fonts (all are variations of Hershey)
Identifier Description
cv::FONT_HERSHEY_SIMPLEX Normal size sans-serif
cv::FONT_HERSHEY_PLAIN Small size sans-serif
cv::FONT_HERSHEY_DUPLEX Normal size sans-serif; more complex than cv::FONT_HERSHEY_​SIM⁠PLEX
cv::FONT_HERSHEY_COMPLEX Normal size serif; more complex than cv::FONT_HERSHEY_DUPLEX
cv::FONT_HERSHEY_TRIPLEX Normal size serif; more complex than cv::FONT_HERSHEY_COMPLEX
cv::FONT_HERSHEY_COMPLEX_SMALL Smaller version of cv::FONT_HERSHEY_COMPLEX
cv::FONT_HERSHEY_SCRIPT_SIMPLEX Handwriting style
cv::FONT_HERSHEY_SCRIPT_COMPLEX More complex variant of cv::FONT_HERSHEY_SCRIPT_SIMPLEX

Any of the font names listed in Table 6-3 can also be combined (via an OR operator) with cv::FONT_HERSHEY_ITALIC to render the indicated font in italics. Each font has a “natural” size. When fontScale is not 1.0, then the font size is scaled by this number before the text is drawn. Figure 6-3 shows a sample of each font.

The eight fonts of Table 6-3, with the origin of each line separated from the vertical by 30 pixels
Figure 6-3. The eight fonts of Table 6-3, with the origin of each line separated from the vertical by 30 pixels

cv::getTextSize()

cv::Size cv::getTextSize(
  const string& text,
  cv::Point     origin,
  int           fontFace,
  double        fontScale,
  int           thickness,
  int*          baseLine
);

The cv::getTextSize() function answers the question of how big some text would be if you were to draw it (with some set of parameters) without actually drawing it on an image. The only novel argument to cv::getTextSize() is baseLine, which is actually an output parameter. baseLine is the y-coordinate of the text baseline relative to the bottommost point in the text.3

Summary

In this short chapter we learned a few new functions that we can use to draw on and annotate images. These functions all operate on the same cv::Mat image types we have been using in the prior chapters. Most of these functions have very similar interfaces, allowing us to draw lines and curves of various thicknesses and colors. In addition to lines and curves, we also saw how OpenCV handles writing text onto an image. All of these functions are extremely useful, in practice, when we are debugging code, as well as for displaying results of our computations on top of the images they are using for input.

Exercises

For the following exercises, modify the code of Example 2-1 to get an image displayed, or modify the code from Example 2-3 to load and display video or camera images.

  1. Drawing practice: load or create and display a color image. Draw one example of every shape and line that OpenCV can draw.

  2. Grayscale: load and display a color image.

    1. Turn it into three-channel grayscale (it is still an BGR image, but it looks gray to the user).

    2. Draw color text onto the image.

  3. Dynamic text: load and display video from a video or from a camera.

    1. Draw a frame per second (FPS) counter somewhere on the image.

  4. Make a drawing program. Load or create an image and display it.

    1. Allow a user to draw a basic face.

    2. Make the components of the face editable (you will have to maintain a list of what was drawn, and when it is altered you might erase and redraw it the new size).

  5. Use cv::LineIterator to count pixels on different line segments in, say, a 300 × 300 image.

    1. At what angles do you get the same number of pixels for 4-connected and 8-connected lines?

    2. For line segment angles other than the above, which counts more pixels: 4-connected or 8-connected lines?

    3. For a given line segment, explain the difference in the length of the line compared to the number of pixels you count iterating along the line for both 4-connected and 8-connected? Which connectedness is closer to the true line length?

1 There is a slightly confusing point here, which is mostly due to legacy in origin. The macro CV_RGB(r,g,b) produces a cv::Scalar s with value s.val[] = { b, g, r, 0 }. This is as it should be, as general OpenCV functions know what is red, green, or blue only by the order, and the ordering convention for image data is BGR as stated in the text.

2 The algorithm used by cv::fillComvexPoly() is actually somewhat more general than implied here. It will correctly draw any polygon whose contour intersects every horizontal line at most twice (though it is allowed for the top or bottom of the polygon to be flat with respect to the horizontal). Such a polygon is said to be “monotone with respect to the horizontal.”

3 The “baseline” is the line on which the bottoms of characters such as a and b are aligned. Characters such as y and g hang below the baseline.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset