In this chapter, we will learn how to use computer vision algorithms for performing advanced video analysis and processing using the ofxOpenCv addon and the OpenCV library. You will learn how to work with the addon's class of images and perform filtering, geometrical transformations of images, and find contours of the objects in the image. Finally, we will consider how to use native OpenCV functions in an example and also learn to use optical flow. We will cover the following topics:
- Using ofxOpenCv
- Motion detection
- Image filtering
- Geometrical transformations of images
- Searching for objects in an image
- Optical flow
- Video morphing
Computer vision is a wide scientific field between mathematics and computer sciences. Its primary goal is to build automatic methods for understanding the content of images. This goal is difficult to achieve; however, we already have a lot of great algorithms, including image enhancement and correction, object detection, tracking and recognition, stereo vision, and automatic machine learning.
You will find that many of the algorithms such as image filtering and object tracking are common for computer vision and video processing. The reason is that some basic procedures are common while dealing with images. The difference is in the goal. The goal of computer vision is automatic analysis of images from the cameras used for controlling machines; for example, robots, or interactive installations. The goal of video processing is in creating video and video effects for spectators to watch.
If you want to use existing computer vision methods, a good choice is Open Computer Vision (OpenCV) library. It contains hundreds of classic and newest algorithms for image processing and analysis. Currently, the library is going through the standardization stage by Khronos Group (khronos.org), and it will be standard like OpenGL and OpenCL soon.
Tip
Though OpenCV works with stereo vision and has functions for transforming 3D point clouds, it's mainly focused on working with two-dimensional images obtained from cameras. So, for processing and analyzing 3D point clouds that are obtained from depth cameras, you should use your own algorithms. Also, you can use the PCL library, which is a special library for working with 3D point clouds. However, note that this is outside the scope of this book.
OpenCV has distinct functions for working with CPU and GPU. We will only consider the CPU functions because these are simpler to learn. Always remember that OpenCV functions are highly optimized so they always work faster than your own pixel-by-pixel algorithms. However, CPU functions are not fast enough for processing big images such as Full HD frames in real time. For such purposes, you need to use GPU. If you need just image processing with some video effects, you may use the shaders technology without OpenCV at all because using shaders is normally simpler and more universal. However, when real computer vision stuff is needed, then use OpenCV's GPU functions.
The way to start linking OpenCV to the openFrameworks project is to use the ofxOpenCv addon. It also adds a number of classes which simplify the common tasks of image processing and tracking. Most of this chapter will be devoted to ofxOpenCv and only the last section Using OpenCV functions will be devoted to using OpenCV functions directly.