cv::resize()

The cv::resize() function handles all of these resizing needs. We provide our input image and the size we would like it converted to, and it will generate a new image of the desired size.

void cv::resize(
  cv::InputArray  src,                             // Input image
  cv::OutputArray dst,                             // Result image
  cv::Size        dsize,                           // New size
  double          fx            = 0,               // x-rescale
  double          fy            = 0,               // y-rescale
  int             interpolation = CV::INTER_LINEAR // interpolation method
);

We can specify the size of the output image in two ways. One way is to use absolute sizing; in this case, the dsize argument directly sets the size we would like the result image dst to be. The other option is to use relative sizing; in this case, we set dsize to cv::Size(0,0), and set fx and fy to the scale factors we would like to apply to the x- and y-axes, respectively.¹ The last argument is the interpolation method, which defaults to linear interpolation. The other available options are shown in Table 11-1.

Interpolation is an important issue here. Pixels in the source image sit on an integer grid; for example, we can refer to a pixel at location (20, 17). When these integer locations are mapped to a new image, there can be gaps—either because the integer source pixel locations are mapped to float locations in the destination image and must be rounded to the nearest integer pixel location, or because there are some locations to which no pixels are mapped (think about doubling the image size by stretching it; then every other destination pixel would be left blank). These problems are generally referred to as forward projection problems. To deal with such rounding problems and destination gaps, we actually solve the problem backward: we step through each pixel of the destination image and ask, “Which pixels in the source are needed to fill in this destination pixel?” These source pixels will almost always be on fractional pixel locations, so we must interpolate the source pixels to derive the correct value for our destination value. The default method is bilinear interpolation, but you may choose other methods (as shown in Table 11-1).

The easiest approach is to take the resized pixel’s value from its closest pixel in the source image; this is the effect of choosing the interpolation value cv::INTER_NEAREST. Alternatively, we can linearly weight the 2 × 2 surrounding source pixel values according to how close they are to the destination pixel, which is what cv::INTER_LINEAR does. We can also virtually place the new, resized pixel over the old pixels and then average the covered pixel values, as done with cv::INTER_AREA.² For yet smoother interpolation, we have the option of fitting a cubic spline between the 4 × 4 surrounding pixels in the source image and then reading off the corresponding destination value from the fitted spline; this is the result of choosing the cv::INTER_CUBIC interpolation method. Finally, we have the Lanczos interpolation, which is similar to the cubic method, but uses information from an 8 × 8 area around the pixel.³

cv::pyrDown()

Normally, we produce layer (i + 1) in the Gaussian pyramid (we denote this layer G_i+1) from layer G_i of the pyramid, by first convolving G_i with a Gaussian kernel and then removing every even-numbered row and column. Of course, in this case, it follows that each image is exactly one-quarter the area of its predecessor. Iterating this process on the input image G₀ produces the entire pyramid. OpenCV provides us with a method for generating each pyramid stage from its predecessor:

void cv::pyrDown(
  cv::InputArray  src,                         // Input image
  cv::OutputArray dst,                         // Result image
  const cv::Size& dstsize = cv::Size()         // Output image size
);

The cv::pyrDown() method will do exactly this for us if we leave the destination size argument dstsize set to its default value of cv::Size(). To be a little more specific, the default size of the output image is ( (src.cols+1)/2, (src.rows+1)/2 ).⁴ Alternatively, we can supply a dstsize, which will indicate the size we would like for the output image; dstsize, however, must obey some very strict constraints. Specifically:

This restriction means that the destination image is very close to half the size of the source image. The dstsize argument is used only for handling somewhat esoteric cases in which very tight control is needed on how the pyramid is constructed.

cv::buildPyramid()

It is a relatively common situation that you have an image and wish to build a sequence of new images that are each downscaled from their predecessor. The function cv::buildPyramid() creates such a stack of images for you in a single call.

void cv::buildPyramid(
  cv::InputArray          src,                 // Input image
  cv::OutputArrayOfArrays dst,                 // Output images from pyramid
  int                     maxlevel             // Number of pyramid levels
);

The argument src is the source image. The argument dst is of a somewhat unusual-looking type cv::OutputArrayOfArrays, but you can think of this as just being an STL vector<> of objects of type cv::OutputArray. The most common example of this would be vector<cv::Mat>. The argument maxlevel indicates how many pyramid levels are to be constructed.

The argument maxlevel is any integer greater than or equal to 0, and indicates the number of pyramid images to be generated. When cv::buildPyramid() runs, it will return a vector in dst that is of length maxlevel+1. The first entry in dst will be identical to src. The second will be half as large—that is, as would result from calling cv::pyrDown(). The third will be half the size of the second, and so on (see the lefthand image in of Figure 11-1).

Figure 11-1. An image pyramid generated with maxlevel=3 (left); two pyramids interleaved together to create a pyramid (right)

cv::pyrUp()

Similarly, we can convert an existing image to an image that is twice as large in each direction by the following analogous (but not inverse!) operation:

void cv::pyrUp(
  cv::InputArray  src,                         // Input image
  cv::OutputArray dst,                         // Result image
  const cv::Size& dstsize = cv::Size()         // Output image size
);

In this case, the image is first upsized to twice the original in each dimension, with the new (even) rows filled with 0s. Thereafter, a convolution is performed with the Gaussian filter⁵ to approximate the values of the “missing” pixels.

Analogous to cv::PyrDown(), if dstsize is set to its default value of cv::Size(), the resulting image will be exactly twice the size (in each dimension) as src. Again, we can supply a dstsize that will indicate the size we would like for the output image dstsize, but it must again obey some very strict constraints. Specifically:

This restriction means that the destination image is very close to double the size of the source image. As before, the dstsize argument is used only for handling somewhat esoteric cases in which very tight control is needed over how the pyramid is constructed.

The Laplacian pyramid

We noted previously that the operator cv::pyrUp() is not the inverse of cv::pyrDown().  This should  be evident because cv::pyrDown() is an operator that loses information. In order to restore the original (higher-resolution) image, we would require access to the information that was discarded by the downsampling process. This data forms the Laplacian pyramid. The ith layer of the Laplacian pyramid is defined by the relation:

Here the operator UP() upsizes by mapping each pixel in location (x, y) in the original image to pixel (2x + 1, 2y + 1) in the destination image; the ⊗ symbol denotes convolution; and g_5×5 is a 5 × 5 Gaussian kernel. Of course, UP(G_i+1)⊗g_5×5 is the definition of the cv::pyrUp() operator provided by OpenCV. Hence, we can use OpenCV to compute the Laplacian operator directly as:

The Gaussian and Laplacian pyramids are shown diagrammatically in Figure 11-2, which also shows the inverse process for recovering the original image from the subimages. Note how the Laplacian is really an approximation that uses the difference of Gaussians, as revealed in the preceding equation and diagrammed in the figure.

Figure 11-2. The Gaussian pyramid and its inverse, the Laplacian pyramid

cv::warpAffine(): Dense affine transformations

In the first case, the obvious input and output formats are images, and the implicit requirement is that the warping assumes the pixels are a dense representation of the underlying image. This means that image warping must necessarily handle interpolations so that the output images are smooth and look natural. The affine transformation function provided by OpenCV for dense transformations is cv::warpAffine():

void cv::warpAffine(
  cv::InputArray    src,                               // Input image
  cv::OutputArray   dst,                               // Result image
  cv::InputArray    M,                                 // 2-by-3 transform mtx
  cv::Size          dsize,                             // Destination image size
  int               flags       = cv::INTER_LINEAR,    // Interpolation, inverse
  int               borderMode  = cv::BORDER_CONSTANT, // Pixel extrapolation
  const cv::Scalar& borderValue = cv::Scalar()         // For constant borders
);

Here src and dst are your source and destination arrays, respectively. The input M is the 2 × 3 matrix we introduced earlier that quantifies the desired transformation. Each element in the destination array is computed from the element of the source array at the location given by:

In general, however, the location indicated by the righthand side of this equation will not be an integer pixel. In this case, it is necessary to use interpolation to find an appropriate value for dst(x, y). The next argument, flags, selects the interpolation method. The available interpolation methods are those in Table 11-1, the same as cv::resize(), plus one additional option, cv::WARP_INVERSE_MAP (which may be added with the usual Boolean OR). This option is a convenience that allows for inverse warping from dst to src instead of from src to dst. The final two arguments are for border extrapolation, and have the same meaning as similar arguments in image convolutions (see Chapter 10).

cv::getAffineTransform(): Computing an affine map matrix

OpenCV provides two functions to help you generate the map matrix M. The first is used when you already have two images that you know to be related by an affine transformation or that you’d like to approximate in that way:

cv::Mat cv::getAffineTransform(              // Return 2-by-3 matrix
  const cv::Point2f* src,                    // Coordinates *three* of vertices
  const cv::Point2f* dst                     // Target coords, three vertices
);

Here src and dst are arrays containing three two-dimensional (x, y) points. The return value is an array that is the affine transform computed from those points.

In essence, the arrays of points in src and dst in cv::getAffineTransform() define two parallelograms. The points in src will be mapped by an application cv::warpAffine(), using the resulting matrix M, to the corresponding points in dst; all other points will be dragged along for the ride. Once these three independent corners are mapped, the mapping of all of the other points is completely determined.

Example 11-1 shows some code that uses these functions.  In the example, we obtain the cv::warpAffine() matrix parameters by first constructing two three-component arrays of points (the corners of our representative parallelogram) and then convert that to the actual transformation matrix using cv::getAffineTransform(). We then do an affine warp followed by a rotation of the image. For our array of representative points in the source image, called srcTri[], we take the three points: (0,0), (0,height-1), and (width-1,0). We then specify the locations to which these points will be mapped in the corresponding array dstTri[].

#include <opencv2/opencv.hpp>
#include <iostream>

using namespace std;

int main(int argc, char** argv) {

  if(argc != 2) {
    cout << "Warp affine
Usage: " <<argv[0] <<" <imagename>
" << endl;
    return -1;
  }

  cv::Mat src = cv::imread(argv[1],1);
  if( src.empty() ) { cout << "Can not load " << argv[1] << endl; return -1; }

  cv::Point3f srcTri[] = {
     cv::Point2f(0,0),           // src Top left
     cv::Point2f(src.cols-1, 0), // src Top right
     cv::Point2f(0, src.rows-1)  // src Bottom left
  };

  cv::Point2f dstTri[] = {
     cv::Point2f(src.cols*0.f,   src.rows*0.33f), // dst Top left
     cv::Point2f(src.cols*0.85f, src.rows*0.25f), // dst Top right
     cv::Point2f(src.cols*0.15f, src.rows*0.7f)   // dst Bottom left
  };

  // COMPUTE AFFINE MATRIX
  //
  cv::Mat warp_mat = cv::getAffineTransform(srcTri, dstTri);
  cv::Mat dst, dst2;
  cv::warpAffine(
    src,
    dst,
    warp_mat,
    src.size(),
    cv::INTER_LINEAR,
    cv::BORDER_CONSTANT,
    cv::Scalar()
  );
  for( int i = 0; i < 3; ++i )
    cv::circle(dst, dstTri[i], 5, cv::Scalar(255, 0, 255), -1, cv::AA);

  cv::imshow("Affine Transform Test", dst);
  cv::waitKey();

  for(int frame=0;;++frame) {

    // COMPUTE ROTATION MATRIX
    cv::Point2f center(src.cols*0.5f, src.rows*0.5f);
    double angle = frame*3 % 360, scale = (cos((angle - 60)* cv::PI/180) + 1.05)*0.8;

    cv::Mat rot_mat = cv::getRotationMatrix2D(center, angle, scale);

    cv::warpAffine(
      src,
      dst,
      rot_mat,
      src.size(),
      cv::INTER_LINEAR,
      cv::BORDER_CONSTANT,
      cv::Scalar()
    );
    cv::imshow("Rotated Image", dst);
    if(cv::waitKey(30) >= 0 )
      break;

  }

  return 0;
}

The second way to compute the map matrix M is to use cv::getRotationMatrix2D(), which computes the map matrix for a rotation around some arbitrary point, combined with an optional  rescaling. This is just one possible kind of affine transformation, so it is less general than is cv::getAffineTransform(), but it represents an important subset that has an alternative (and more intuitive) representation that’s easier to work with in your head:

cv::Mat cv::getRotationMatrix2D(               // Return 2-by-3 matrix
  cv::Point2f  center                          // Center of rotation
  double       angle,                          // Angle of rotation
  double       scale                           // Rescale after rotation
);

The first argument, center, is the center point of the rotation. The next two arguments give the magnitude of the rotation and the overall rescaling. The function returns the map matrix M, which (as always) is a 2 × 3 matrix of floating-point numbers.

If we define α = scale * cos(angle) and β = scale * sin(angle), then this function computes the matrix M to be:

You can combine these methods of setting the map matrix M to obtain, for example, an image that is rotated, scaled, and warped.

cv::transform(): Sparse affine transformations

We have explained that cv::warpAffine() is the right way to handle dense mappings. For sparse mappings (i.e., mappings of lists of individual points), it is best to use cv::transform(). You will recall from Chapter 5 that the transform method has the following prototype:

void cv::transform(
  cv::InputArray  src,                     // Input N-by-1 array (Ds channels)
  cv::OutputArray dst,                     // Output N-by-1 array (Dd channels)
  cv::InputArray  mtx                      // Transform matrix (Ds-by-Dd)
);

In general, src is an N × 1 array with D_s channels, where N is the number of points to be transformed and D_s is the dimension of those source points. The output array dst will be the same size but may have a different number of channels, D_d. The transformation matrix mtx is a D_s × D_d matrix that is then applied to every element of src, after which the results are placed into dst.

In the case of transformations that are simple rotations in this channel space, our transformation matrix mtx will be a 2 × 2 matrix only, and it can be applied directly to the two-channel indices of src. In fact this is true for rotations, stretching, and warping as well in some simple cases. Usually, however, to do a general affine transformation (including translations and rotations about arbitrary centers, and so on), it is necessary to extend the number of channels in src to three, so that the action of the more usual 2 × 3 affine transformation matrix is defined. In this case, all of the third-channel entries must be set to 1 (i.e., the points must be supplied in homogeneous coordinates). Of course, the output array will still be a two-channel array.

cv::invertAffineTransform(): Inverting an affine transformation

Given an affine transformation represented as a 2 × 3 matrix, it is often desirable to be able to compute the inverse transformation, which can be used to “put back” all of the transformed points to where they came from. This is done with cv::invertAffineTransform():

void cv::invertAffineTransform(
  cv::InputArray  M,                           // Input 2-by-3 matrix
  cv::OutputArray iM                           // Output also a 2-by-3 matrix
);

This function takes a 2 × 3 array M and returns another 2 × 3 array iM that inverts M. Note that cv::invertAffineTransform() does not actually act on any image, it just supplies the inverse transform. Once you have iM, you can use it as you would have used M, with either cv::warpAffine() or cv::transform(). 

cv::warpPerspective(): Dense perspective transform

The dense perspective transform uses an OpenCV function that is analogous to the one provided for dense affine transformations. Specifically, cv::warpPerspective() has all of the same arguments as cv::warpAffine(), except with the small, but crucial, distinction that the map matrix must now be 3 × 3.

void cv::warpPerspective(
  cv::InputArray    src,                               // Input image
  cv::OutputArray   dst,                               // Result image
  cv::InputArray    M,                                 // 3-by-3 transform mtx
  cv::Size          dsize,                             // Destination image size
  int               flags       = cv::INTER_LINEAR,    // Interpolation, inverse
  int               borderMode  = cv::BORDER_CONSTANT, // Extrapolation method
  const cv::Scalar& borderValue = cv::Scalar()         // For constant borders
);

Each element in the destination array is computed from the element of the source array at the location given by:

As with the affine transformation, the location indicated by the right side of this equation will not (generally) be an integer location. Again the flags argument is used to select the desired interpolation method, and has the same possible values as the corresponding argument to cv::warpAffine().

cv::getPerspectiveTransform(): Computing the perspective map matrix

As with the affine transformation, for filling the map matrix in the preceding code, we have a convenience function that can compute the transformation matrix from a list of point correspondences:

cv::Mat cv::getPerspectiveTransform(           // Return 3-by-3 matrix
  const cv::Point2f* src,                      // Coordinates of *four* vertices
  const cv::Point2f* dst                       // Target coords, four vertices
);

The src and dst argument are now arrays of four (not three) points, so we can independently control how the corners of (typically) a rectangle in src are mapped to (generally) some rhombus in dst. Our transformation is completely defined by the specified destinations of the four source points. As mentioned earlier, for perspective transformations, the return value will be a 3 × 3 array; see Example 11-2 for sample code. Other than the 3 × 3 matrix and the shift from three to four control points, the perspective transformation is otherwise exactly analogous to the affine transformation we already introduced.

#include <opencv2/opencv.hpp>
#include <iostream>

using namespace std;

int main(int argc, char** argv) {

  if(argc != 2) {
    cout << "Perspective Warp
Usage: " <<argv[0] <<" <imagename>
" << endl;
    return -1;
  }

  Mat src = cv::imread(argv[1],1);
  if( src.empty() ) { cout << "Can not load " << argv[1] << endl; return -1; }

  cv::Point2f srcQuad[] = {
    cv::Point2f(0,          0),          // src Top left
    cv::Point2f(src.cols-1, 0),          // src Top right
    cv::Point2f(src.cols-1, src.rows-1), // src Bottom right
    cv::Point2f(0,          src.rows-1)  // src Bottom left
  };

  cv::Point2f dstQuad[] = {
    cv::Point2f(src.cols*0.05f, src.rows*0.33f),
    cv::Point2f(src.cols*0.9f,  src.rows*0.25f),
    cv::Point2f(src.cols*0.8f,  src.rows*0.9f),
    cv::Point2f(src.cols*0.2f,  src.rows*0.7f)
  };

  // COMPUTE PERSPECTIVE MATRIX
  //
  cv::Mat warp_mat = cv::getPerspectiveTransform(srcQuad, dstQuad);
  cv::Mat dst;
  cv::warpPerspective(src, dst, warp_mat, src.size(), cv::INTER_LINEAR,
                      cv::BORDER_CONSTANT, cv::Scalar());

  for( int i = 0; i < 4; i++ )
    cv::circle(dst, dstQuad[i], 5, cv::Scalar(255, 0, 255), -1, cv::AA);

  cv::imshow("Perspective Transform Test", dst);
  cv::waitKey();
  return 0;

}

cv::perspectiveTransform(): Sparse perspective transformations

cv::perspectiveTransform() is a special function that performs perspective transformations on lists of points. Because cv::transform() is limited to linear operations, it cannot properly handle perspective transforms. This is because such transformations require division by the third coordinate of the homogeneous representation (x = f *X/Z,y = f * Y/Z). The special function cv::perspectiveTransform() takes care of this for us:

void cv::perspectiveTransform(
  cv::InputArray  src,                 // Input N-by-1 array (2 or 3 channels)
  cv::OutputArray dst,                 // Output N-by-1 array (2 or 3 channels)
  cv::InputArray  mtx                  // Transform matrix (3-by-3 or 4-by-4)
);

As usual, the src and dst arguments are, respectively, the array of source points to be transformed and the array of destination points resulting from the transformation. These arrays should be two- or three-channel arrays. The matrix mtx can be either a 3 × 3 or a 4 × 4 matrix. If it is 3 × 3, then the projection is from two dimensions to two; if the matrix is 4 × 4, then the projection is from three dimensions to three.

In the current context, we are transforming a set of points in an image to another set of points in an image, which sounds like a mapping from two dimensions to two dimensions. This is not exactly correct, however, because the perspective transformation is actually mapping points on a two-dimensional plane embedded in a three-dimensional space back down to a (different) two-dimensional subspace. Think of this as being just what a camera does. (We will return to this topic in greater detail when discussing cameras in later chapters.) The camera takes points in three dimensions and maps them to the two dimensions of the camera imager. This is essentially what is meant when the source points are taken to be in “homogeneous coordinates.” We are adding a dimension to those points by introducing the Z dimension and then setting all of the Z values to 1. The projective transformation is then projecting back out of that space onto the two-dimensional space of our output. This is a rather long-winded way of explaining why, when mapping points in one image to points in another, you will need a 3 × 3 matrix.

Outputs of the code in Examples 11-1 and 11-2 are shown in Figure 11-4 for affine and perspective transformations. In these examples, we transform actual images; you can compare these with the simple diagrams back in Figure 11-3.

Figure 11-4. Perspective and affine mapping of an image

cv::cartToPolar(): Converting from Cartesian to polar coordinates

To map from Cartesian coordinates to polar coordinates, we use the function cv::cartToPolar():

void cv::cartToPolar(
  cv::InputArray  x,                         // Input single channel x-array
  cv::InputArray  y,                         // Input single channel y-array
  cv::OutputArray magnitude,                 // Output single channel mag-array
  cv::OutputArray angle,                     // Output single channel angle-array
  bool            angleInDegrees = false     // Set true for degrees, else radians
);

The first two arguments x, and y, are single-channel arrays. Conceptually, what is being represented here is not just a list of points, but a vector field¹⁰—with the x-component of the vector field at any given point being represented by the value of the array x at that point, and the y-component of the vector field at any given point being represented by the value of the array y at that point. Similarly, the result of this function appears in the arrays magnitude and angle, with each point in magnitude representing the length of the vector at that point in x and y, and each point in angle representing the orientation of that vector. The angles recorded in angle will, by default, be in radians—that is, [0, 2π). If the argument angleInDegrees is set to true, however, then the angles array will be recorded in degrees [0, 360). Also note that the angles are computed using (approximately) atan2(y,x), so an angle of 0 corresponds to a vector pointing in the direction.

As an example of where you might use this function, suppose you have already taken the x- and y-derivatives of an image, either by using cv::Sobel() or by using convolution functions via cv::DFT() or cv::filter2D(). If you stored the x-derivatives in an image dx_img and the y-derivatives in dy_img, you could now create an edge-angle recognition histogram; that is, you could collect all the angles provided the magnitude or strength of the edge pixel is above some desired threshold. To calculate this, we would first create two new destination images (and call them img_mag and img_angle, for example) for the directional derivatives and then use the function cv::cartToPolar( dx_img, dy_img, img_mag, img_angle, 1 ). We would then fill a histogram from img_angle as long as the corresponding “pixel” in img_mag is above our desired threshold.

cv::polarToCart(): Converting from polar to Cartesian coordinates

The function cv::cartToPolar() performs the reverse mapping from polar coordinates to Cartesian coordinates.

void cv::polarToCart(
  cv::InputArray  magnitude,               // Output single channel mag-array
  cv::InputArray  angle,                   // Output single channel angle-array
  cv::OutputArray x,                       // Input single channel x-array
  cv::OutputArray y,                       // Input single channel y-array
  bool            angleInDegrees = false   // Set true for degrees, else radians
);

The inverse operation is also often useful, allowing us to convert from polar back to Cartesian coordinates. It takes essentially the same arguments as cv::cartToPolar(), with the exception that magnitude and angle are now inputs, and x and y are now the results.

cv::logPolar()

The OpenCV function for a log-polar transform is cv::logPolar():

void cv::logPolar(
  cv::InputArray  src,                          // Input image
  cv::OutputArray dst,                          // Output image
  cv::Point2f     center,                       // Center of transform
  double          m,                            // Scale factor
  int             flags = cv::INTER_LINEAR      // interpolation and fill modes
                        | cv::WARP_FILL_OUTLIERS
);

The src and dst are the usual input and output images. The parameter center is the center point (x_c, y_c) of the log-polar transform; m is the scale factor, which should be set so that the features of interest dominate the available image area. The flags parameter allows for different interpolation methods. The interpolation methods are the same set of standard interpolations available in OpenCV (see Table 11-1). The interpolation methods can be combined with either or both of the flags CV::WARP_FILL_OUTLIERS (to fill points that would otherwise be undefined) or CV::WARP_INVERSE_MAP (to compute the reverse mapping from log-polar to Cartesian coordinates).

Sample log-polar coding is given in Example 11-3, which demonstrates the forward and backward (inverse) log-polar transform. The results on a photographic image are shown in Figure 11-7.

Figure 11-7. Log-polar example on an elk with transform centered at the white circle on the left; the output is on the right

#include <opencv2/opencv.hpp>
#include <iostream>

using namespace std;

int main(int argc, char** argv) {

  if(argc != 3) {
    cout << "LogPolar
Usage: " <<argv[0] <<" <imagename> <M value>
"
      <<"<M value>~30 is usually good enough
";
    return -1;
  }

  cv::Mat src = cv::imread(argv[1],1);

  if( src.empty() ) { cout << "Can not load " << argv[1] << endl; return -1; }

  double M = atof(argv[2]);
  cv::Mat dst(src.size(), src.type()), src2(src.size(), src.type());

  cv::logPolar(
    src,
    dst,
    cv::Point2f(src.cols*0.5f, src.rows*0.5f),
    M,
    cv::INTER_LINEAR | cv::WARP_FILL_OUTLIERS
  );
  cv::logPolar(
    dst,
    src2,
    cv::Point2f(src.cols*0.5f, src.rows*0.5f),
    M,
    cv::INTER_LINEAR | cv::WARP_INVERSE_MAP
  );
  cv::imshow( "log-polar", dst );
  cv::imshow( "inverse log-polar", src2 );

  cv::waitKey();

  return 0;
}

cv::remap(): General image remapping

void cv::remap(
  cv::InputArray    src,                                 // Input image
  cv::OutputArray   dst,                                 // Output image
  cv::InputArray    map1,                                // target x for src pix
  cv::InputArray    map2,                                // target y for src pix
  int               interpolation = cv::INTER_LINEAR,    // Interpolation, inverse
  int               borderMode    = cv::BORDER_CONSTANT, // Extrapolation method
  const cv::Scalar& borderValue   = cv::Scalar()         // For constant borders
);

The first two arguments of cv::remap() are the source and destination images, respectively. The next two arguments, map1 and map2, indicate the target x and y locations, respectively, where any particular pixel is to be relocated. This is how you specify your own general mapping. These should be the same size as the source and destination images, and must be one of the following data types: CV::S16C2, CV::F32C1, or CV::F32C2. Noninteger mappings are allowed: cv::remap() will do the interpolation calculations for you automatically.

The next argument, interpolation, contains flags that tell cv::remap() exactly how that interpolation is to be done. Any one of the values listed in Table 11-1 will work—except for cv::INTER_AREA, which is not implemented  for cv::remap(). 

Basic FNLMD with cv::fastNlMeansDenoising()

void cv::fastNlMeansDenoising(
  cv::InputArray  src,                    // Input image
  cv::OutputArray dst,                    // Output image
  float           h                  = 3, // Weight decay parameter
  int             templateWindowSize = 7, // Size of patches used for comparison
  int             searchWindowSize   = 21 // Maximum patch distance to consider
);

The first of these four functions, cv::fastNlMeansDenoising(), implements the algorithm as described exactly. We compute the result array dst from the input array src using a patch area of templateWindowSize and a decay parameter of h, and patches inside of searchWindowSize distance are considered.  The image may be one-, two-, or three-channel, but must be or type cv::U8.¹³ Table 11-2 lists some values, provided  from the authors of the algorithm, that you can use to help set the decay parameter, h.

Table 11-2. Recommended values for cv::fastNlMeansDenoising() for grayscale images

Noise: σ	Patch size: s	Search window	Decay parameter: h
	3 × 3	21 × 21	0.40 · σ
	5 × 5	21 × 21	0.40 · σ
	7 × 7	35 × 35	0.35 · σ
	9 × 9	35 × 35	0.35 · σ
	11 × 11	35 × 35	0.30 · σ

FNLMD on color images with cv::fastNlMeansDenoisingColor()

void cv::fastNlMeansDenoisingColored(
  cv::InputArray  src,                    // Input image
  cv::OutputArray dst,                    // Output image
  float           h                  = 3, // Luminosity weight decay parameter
  float           hColor             = 3, // Color weight decay parameter
  int             templateWindowSize = 7, // Size of patches used for comparison
  int             searchWindowSize   = 21 // Maximum patch distance to consider
);

The second variation of the FNLMD algorithm is used for color images. It accepts only images of type cv::U8C3. Though it would be possible in principle to apply the algorithm more or less directly to an RGB image, in practice it is better to convert the image to a different color space for the computation. The function cv::fastNlMeansDenoisingColored() first converts the image to the LAB color space, then applies the FNLMD algorithm, then converts the result back to RGB. The primary advantage of this is that in color there are, in effect, three decay parameters. In an RGB representation, however, it would be unlikely that you would want to set any of them to distinct values. But in the LAB space, it is natural to assign a different decay parameter to the luminosity component than to the color components. The function cv::fastNlMeansDenoisingColored() allows you to do just that. The parameter h is used for the luminosity decay parameter, while the new parameter hColor is used for the color channels. In general, the value of hColor will be quite a bit smaller than h. In most contexts, 10 is a suitable value. Table 11-3 lists some values that you can use to help set the decay parameter, h.

Table 11-3. Recommended values for cv::fastNlMeansDenoising() for color images

Noise: σ	Patch size: s	Search window	Decay parameter: h
	3 × 3	21 × 21	0.55 · σ
	5 × 5	35 × 35	0.40 · σ
	7 × 7	35 × 35	0.35 · σ

FNLMD on video with cv::fastNlMeansDenoisingMulti() and cv::fastNlMeansDenoisingColorMulti()

void cv::fastNlMeansDenoisingMulti(
  cv::InputArrayOfArrays srcImgs,                // Sequence of several images
  cv::OutputArray        dst,                    // Output image
  int                    imgToDenoiseIndex,      // Index of image to denoise
  int                    temporalWindowSize,     // Num images to use (odd)
  float                  h                  = 3, // Weight decay parameter
  int                    templateWindowSize = 7, // Size of comparison patches
  int                    searchWindowSize   = 21 // Maximum patch distance
);
void cv::fastNlMeansDenoisingColoredMulti(
  cv::InputArrayOfArrays srcImgs,                // Sequence of several images
  cv::OutputArray        dst,                    // Output image
  int                    imgToDenoiseIndex,      // Index of image to denoise
  int                    temporalWindowSize,     // Num images to use (odd)
  float                  h                  = 3, // Weight decay param
  float                  hColor             = 3, // Weight decay param for color
  int                    templateWindowSize = 7, // Size of comparison patches
  int                    searchWindowSize   = 21 // Maximum patch distance
);

The third and fourth variations are used for sequential images, such as those that might be captured from video. In the case of sequential images, it is natural to imagine that frames other than just the current one might contain useful information for denoising a pixel. In most applications the noise will not be constant between images, while the signal will likely be similar or even identical. The functions cv::fastNlMeansDenoisingMulti() and cv::fastNlMeansDenoisingColorMulti() expect an  array of images, srcImgs, rather than a single image. Additionally,  they must be told which image in the sequence is actually to be denoised; this is done with the parameter imgToDenoiseIndex. Finally, a temporal window must be provided that indicates the number of images from the sequence to be used in the denoising. This parameter must be odd, and the implied window is always centered on imgToDenoiseIndex. (Thus, if you were to set imgToDenoiseIndex to 4 and temporalWindowSize to 5, the images that would be used in the denoising would be 2, 3, 4, 5, and 6.)