GrabCut is an excellent iterative background/foreground segmentation algorithm that is available since Version 2.1 of OpenCV. GrabCut is especially useful to separate objects from the background with minimal additional information (a bounding rectangle is sufficient in most cases). However, it is computationally intensive, and so it is only appropriate to segment still images.

The main function is grabCut(InputArray img, InputOutputArray mask, Rect rect, InputOutputArray bgdModel, InputOutputArray fgdModel, int iterCount, int mode=GC_EVAL). The parameters bgdModel and fgdModel are only used internally by the function (though they have to be declared). The iterCount variable is the number of iterations to be performed. In our experience, few iterations of the algorithm are required to produce good segmentations. The algorithm is aided by a bounding rectangle, a mask image, or both. The option chosen is indicated in the mode parameter, which can be GC_INIT_WITH_RECT, GC_INIT_WITH_MASK, or an OR combination of the two. In the former case, rect defines the rectangle. Pixels outside the rectangle are considered as the obvious background. In the latter case, the mask is an 8-bit image in which pixels may have the following values:

The image mask is also the output image with the resulting segmentation, which is derived using those same previous values. OpenCV includes an example of GrabCut ([opencv_source_code]/samples/cpp/grabcut.cpp) in which the user can draw a bounding rectangle as well as foreground and background pixels.

The following grabcut example uses the algorithm with an initial bounding rectangle and then copies the resulting foreground onto another position in the same image:

#include "opencv2/opencv.hpp"
#include <iostream>

using namespace std;
using namespace cv;

int main(int argc, char *argv[])
{
    // Read original image and clone it
    Mat src = imread("stuff.jpg" );
    Mat tgt = src.clone();

    // Create source window
    namedWindow("Source", WINDOW_AUTOSIZE);

    imshow("Source", src);
    moveWindow("Source", 0,0);

    // GrabCut segmentation
    Rect rectangle(180,279,60,60);  // coin position
    Mat result;                     // segmentation result
    Mat bgModel,fgModel;            // used internally
grabCut(src, result, rectangle, bgModel,fgModel, 1, GC_INIT_WITH_RECT);

    result=(result & GC_FGD);   // leave only obvious foreground

    // Translation operation
    Mat aff=Mat::eye(2,3,CV_32FC1);
    aff.at<float>(0,2)=50;
    warpAffine(tgt, src, aff, result.size());
    warpAffine(result, result, aff, result.size());
    src.copyTo(tgt, result);

    // Show target window
    imshow("Target", tgt);
    moveWindow("Target", src.cols, 0);

    cout << "Press any key to exit...
";
    waitKey(); // Wait for key press
    return 0;
}

The preceding example simply uses a fixed rectangle around the coin in the source image (see the fifth screenshot in this chapter) and performs the segmentation. The result image will contain values between 0 (GC_BGD) and 3 (GC_PR_FGD). The ensuing AND operation is needed to convert values other than GC_FGD to zero and thus get a binary foreground mask. Then, both the source image and the mask are translated by 50 pixels in the horizontal. An affine warping operation is used with an identity matrix in which only the x translation component is changed.

Finally, the translated image is copied onto the target image, using the (also translated) mask. Both source and target images are shown in the following screenshot. Increasing the number of iterations did not have any significant effect in this particular example:

Source and target images in the GrabCut example