The Open Source Computer Vision Library (OpenCV for short [1, 14] is a collection of C and C++ source code and executables that span a wide range of computer vision algorithms. The code is optimized and intended for real-time vision applications. Popular application areas supported by OpenCV algorithms in decreasing order of coverage are human-computer interaction; video security; robotics and image retrieval; and factory inspection/machine vision. Look for a full handbook on OpenCV due from Springer in 2004 [12]. OpenCV supports Windows and Linux, but the code is well behaved and has been ported to many other platforms. Tested compilers include Intel's compiler version 6.0 or higher, MSVC++ 6.0 or higher, Borland C++ 5.5 or higher, and GNU C/C++ 2.95.3 or higher.
OpenCV is distributed under a BSD-style license (see Section 11.6). This license allows for royalty-free commercial or research use with no requirement that the user's code be free or open. Intel Labs (http://www.intel.com/research/mrl/index.htm) media group (http://www.intel.com/research/mrl/research/media.htm) undertook development of this library for the following reasons:
- Computer vision algorithms are being studied for their implications on future computer architecture. Optimized code is needed for these studies.
- We wanted to advance external computer vision research by creating an optimized infrastructure that would avoid the need for others to endlessly reinvent and reoptimize common algorithms.
- We wanted to hasten the adoption of computer vision applications out in the market by lowering the algorithmic "barrier to entry" for firms seeking to add computer vision functionality to their products.
OpenCV's main information Web site is located at http://www.intel.com/research/mrl/research/opencv. This site contains pointers to the user group and the download site.
The user group is at http://groups.yahoo.com/group/OpenCV/ [4]. To join the user group, you must first register (free) for Yahoogroups (http://groups.yahoo.com/) and then sign up for OpenCV at http://groups.yahoo.com/group/OpenCV/join. The user group is the forum for reporting bugs, discussing problems, technical issues, posting/reading calls for papers or jobs, and so on. This is an active group with thousands of members. You control at any time whether you want to read it on the web only (no emails) or get daily summaries or every posting. We do not distribute names or email addresses from the list to outside parties.
The download site is on SourceForge at http://sourceforge.net/projects/opencvlibrary/ [6]. At this site, you can obtain the various release versions of OpenCV for Windows and Linux, documentation files, and the foil set [5] from which many of the images in this chapter were taken.
11.1.1. Installation |
Installation[1] under Windows is straightforward. Download the windows executable installation from the SourceForge site to a directory on your local disk and run it. The script installs OpenCV, registers DirectShow filters, and does other post-installation procedures. After it is finished, you may start using OpenCV; you do not need to compile OpenCV into binaries unless you want to debug files.
[1] Adapted with permission from the INSTALL file that comes with the download [6].
It is also possible to build core OpenCV binaries manually from the source-code distribution for Linux (though the executable installation includes sources as well). To build manually from the Linux source, do the following.
+ Download and unpack the OpenCV-*.tar.gz package somewhere, e.g., C:MySoft(the root folder is referred to further as <opencv_root>). The tree should look like this: <opencv_root> _dsw cv include src make cvaux ... + Add <opencv_root>in to the system path. Under Windows 9x/ME, this is done by modifying autoexec.bat. Under NT/2000/XP, it can be done instantly at MyComputer:—right button click—> Properties->Advanced->Environment Variables. + HighGUI requires graphic libraries by default, so remove HAVE_JPEG, HAVE_TIFF, and HAVE_PNG from preprocessor definitions and libjpeg.lib, libtiff.lib, libpng.lib, and zlib .lib from the linker command line. The resultant HighGUI will be able to read and write most of JPEGs, BMPs, uncompressed TIFFs, PXMs and Sun raster images; capture video from AVI or camera via VFW; and write AVIs via VFW. Building OpenCV from sources --------------------------- You need to have a C/C++ compiler. Below are some variants: === Microsoft Visual C++ (6.0 or higher) === This is a preferred variant, because most of the demos are written for it (i.e., using MFC). * If you plan to build DirectShow filters, acquire and setup DirectX SDK as described in <opencv_root>docsfaq.htm or <opencv_root>cvincludecvstreams.h. * If you plan to build an MIL-enabled version of highgui, set up MIL include and library paths in Developer Studio. * If you plan to build MATLAB wrappers, you need to have MATLAB C/C++ interface libraries and to set up Developer Studio properly. Read opencv/interfaces/matlab/readme .txt for details. * If you are going to build TCLTK demo applications (included with the source package only), you need TCLTK and BWidgets. The easiest way to obtain both is to download and install ActiveTcl from http://www. activestate.com. After installing ActiveTCL, - Add <tcl_root>in to the system path (if installer didn't do it). - Add <tcl_root>include and <tcl_root>lib to Developer Studio search paths ( tools->options->directories). Open <opencv_root>\_dswopencv.dsw. Choose from menu Build->Batch Build->Build. Wait and enjoy. If you want to debug OpenCV directshow filters, register them using regsvr32 (e.g., regsvr32 <opencv_root>inCamShiftd.ax). === Other compilers === In case of other compilers, you still can build the core libraries (cv, cvaux, highgui), algorithmic tests, and samples (<opencv_root>samplesc). The following compilers are supported by default: === Intel compiler 6.0 or greater === Run nmake /f makefile.icl in the root OpenCV folder. Because the produced binaries should be compatible with Visual C++, you can then use the DLLs with VisualC++ -build applications etc. === Borland C++ 5.5 (free) or greater === Run make -f makefile.bcc in the root OpenCV folder. Before running compilation, make sure <BorlandC_root>in is in the system path and <BorlandC_root>incc32.cfg contains -I<BorlandC_root>ccinclude -L<BorlandC_root>cclib -L<BorlandC_root>cclibpsdk (where <BorlandC_root> denotes the root folder of Borland C++ installation). === GNU C/C++ 2.95.3 or greater === Run mingw32-make -f makefile.gcc in the root OpenCV folder. Make sure that <gcc_root>in is in the system path. To build VFW-enabled highgui, read instructions in <opencv_root>otherlibs\_graphics readme.txt.
Run algorithmic tests: <opencv_root>incvtest. exe. This will produce cvtest. sum and cvtest.1st. cvtest.sum should contain all OKs. Or, run samples at <opencv_root>samplesc. (Note that some of the demos need AVI or Camera, e.g., motempl.c.)
How to add support for another compiler |
Look at <opencv_root>utilsgen_make.py. It looks at .dsp files in the specified folders and generates makefiles for all compilers it knows. GCC handling now gets a little bit ugly because the linker cannot handle long lists of files and it wasn't known if it was possible to use temporary inline files instead.
There are no prebuilt binaries for the Linux version (because of different C++-incompatible versions of GCC in different distributions), so you have to build it from sources. The following has been tested on RedHat 8.0 (GCC 3.2) and SuSE 8.0 (GCC 2.95.3). To build fully functional libraries and demos in Linux, use
+ motif (LessTif or OpenMotif) with development files. configure script assumes it is at /usr/X11R6/lib & /usr/X11R6/include/Xm. + libpng, libjpeg and libtiff with development files. + libavcodec from ffmpeg 0.4.6(pre) and headers. Earlier version does not fit because of changed interface and because of the GPL license (newer version is LGPL). However, static linking is still prohibited for non-GPL software (such as OpenCV), so: Get CVS snapshot of ffmpeg from ffmpeg.sourceforge.net ./configure --enable-shared make make install You will then have /usr/local/lib/libavcodec. so and /usr/local/include/ffmpeg/*.h. + For building demo applications only: fltk 1.1.x (1.1.1 is currently the preferred one) with development files. If you do not have it, get it from www.fltk.org. In case of RPM-based distribution it is possible to build fltk RPMs by rpmbuild -ta fItk-x.y.z-source.tar.gz (for RH 8.x) or rpm -ta fItk-x.y.z-source.tar.gz (for others). + For demo applications only: TCL/TK 8.3.x with development files and BWidgets >=1.3.x. Download BWidgets from http://sourceforge.net/projects/tcllib/ Now build OpenCV: ====== 1. If your distribution uses RPM, you may build RPMs via rpmbuild -ta OpenCV-x.y.z.tar.gz" (for RH 8.x) or rpm -ta OpenCV-x.y.z.tar.gz" (for others) where OpenCV-x.y.z.tar.gz should be put into /usr/src/redhat/SOURCES/ or a similar folder. The command will build OpenCV-x.y.z.*.rpm (there is no OpenCV-devel; everything is in one package). Then, install it by rpm -i --nodeps OpenCV-x.y.z.*.rpm nodeps is needed in this version, because it cannot find libavcodec.so, even if it is in the path (a weird bug somewhere). 2. If your distribution does not support RPM, build and install it in the *nix traditional way: ./configure --with-apps # or simply ./configure make make install # as root ldconfig # as root Both 1 and 2 (post-install) The default installation path is /usr/local/lib and /usr/local/include/opencv, so you need to add /usr/local/lib to /etc/ld.so.conf (and run ldconfig afterward).
- Run /usr/local/bin/cvtest.
- Or, compile and run simple C examples at /usr/local/share/opencv/ samples, for example, g++ 'opencv-config -cxxfiags' -o morphology morphology.c 'opencv-config -libs'. Plain gee won't work because of unresolved C++ specific symbols (located in highgui).
- Or, run /usr/local/bin/{cvlkdemocvcsdemocvenvvmdemotk}.
When you have set up OpenCV, the directory organization will be something like the following (slightly different for Linux):
C:Program FilesOpenCV -- Location _dsw -- Build files under Windows apps -- Demo application files bin -- Binaries, dlls, filters ... cv -- Source Code cvaux -- "Experimental" Source Code docs -- Manuals filters -- DirectShow filters interfaces -- Ch interpretive C, MATLAB lib -- Static link libraries otherlibs -- highgui interface and camera support samples -- Simple code usage examples tests -- Source and data for test code utils -- cvinfo code and other utilities
apps: The apps directory contains the following:
C:Program FilesOpenCVapps -- Location CamShiftDemo -- Track color probability distributions Common -- Camera routines cvcsdemo -- Color tracker application cvlkdemo -- Optical flow application HaarFaceDetect -- Viola-Jones Boosted face detection HaarTraining -- Training code for Boosted face detection Hawk -- EiC Interpretive C interface HMMDemo -- Face recognition using HMMs StereoDemo -- Depth detection from stereo correspondence Tracker3dDemo -- 3D tracking with multiple cameras VMDemo -- Interpolated view (morph) between 2 images vmdemotk -- Interpolated view application
Of special interest here is the manual in the Hawk directory, which details some of the functions in HighGUI (see below) that make it easy to display an image or video in a window. One-line routines can be used to attach a menu and/or sliders to the window, which can be used to interactively control OpenCV routines operating in the window. The hidden Markov model (HMM) demo does face recognition using horizontal HMMs across a face feeding into a vertical HMM down a face to do 2D face recognition.
The StereoDemo is a console-based application that uses cvcam for video capture and HighGUI for visualization (both in otherlibs directory). To run this demo, you need two USB cameras compatible with DirectShow that can run simultaneously. Creative WebCam is an example of such a camera. Two different cameras types might be also work (not tested). You will need to register ProxyTrans.ax and SyncFilter.ax DirectShow filters from the opencvin folder using regsvr32. exe utility or opencvinRegisterAll. bat batch command file. The demo lets you calibrate the intrinsic and extrinsic parameters for the two cameras by automatically tracking a checkerboard pattern of known size and then finds stereo correspondence between the two image streams and converts that to a disparity or depth map.
Tracker3dDemo demonstrates the 3D calibration of multiple cameras (two or more) into one coordinate frame and then the 3D tracking of multiple objects from multiple views.
docs: The docs directory contains the HTML-based manual (index.htm, faq.htm), the license file (see Section 11.6), and directories describing some of the applications and tutorial papers.
filters: The filters directory contains code to build the following computer vision filters:
C:Program FilesOpenCVfilters -- Location CalibFilter -- Calibrate camera lens via checkerboard CamShift -- Meanshift robust object tracker Condens -- Condensation particle filter tracker Kalman -- Kalman filter ProxyTrans -- Automatic filter calls your code SyncFilter -- For use with multiple cameras Tracker3dFilter -- 3D tracking with multiple cameras
otherlibs: The otherlibs directory contains the camera connection code (cvcam) and HighGUI which can be used to put up a window with still or video images and attach menus and controls to control video processing within the window.
samples: The samples directory is a good place to look for simple code examples of many of the functions in OpenCV.
OpenCV has two defining characteristics: (1) It spans a very wide range of functions, and (2) the code is fairly well optimized for speed. Figure 11.1 shows what code is open, what is not, and how things relate to one another. As with Intel's other performance libraries [3], when you build your application, you link against a dynamic library file, for example, cv.lib. At runtime, an optimized C dynamic-link library stub, cv.dll is loaded. This stub tries to determine what processor it is running on, and if the processor is identified, it will look for a DLL file that is specifically optimized for that processor. For example, on a Pentium 4, the specially optimized code is OptCVw7.dll. If such a file is found, cv.dll swaps itself out for this file. If the processor type cannot be determined or the corresponding optimized file cannot be found, OpenCV runs optimized C by default.
OpenCV's C code is algorithmically optimized. The processor-specific code is further optimized with inline assembly, memory alignment, and cache considerations, and it takes advantage when possible of the internal SIMD arithmetic MMX and SSE processors. OpenCV no longer depends on Intel's Image Processing Library, but some demos still require it. Again, as long as you have the set of DLLs on your machine, you can just compile/run OpenCV applications and the switcher code inside the cv. dll takes care of loading the correct one and takes advantage of the performance-primitive ippCV code described below.
OpenCV comes from Intel's research labs and is fully open and free to use. Intel has many other software tools that come from other internal teams such as the Software System's Group. This other software is released commercially. Such products include the VTune code profiler, the Intel Compiler, and the multithreading preprocessor macros. Of interest here are Intel's integrated performance primitives (IPP) and specifically the 60 or so computer vision functions included under ippCV. You can get information about IPP at http://developer.intel.com/software/products/ipp/ippvm20/.
The primitives part of the integrated performance primitives refers to functions that operate mostly on vectors, matrices, and simple structures. These functions are re-entrant and stateless. Thus, the IPP functions tend to be easy to interface to and work well embedded in streaming-data systems.
ippCV contains many fewer functions than OpenCV, but the functions are optimized on a wider variety of platforms. The platform support includes
IA32 Intel Itanium Architecture Intel StrongARM Microarchitecture Intel XScale Microarchitecture
Intel IPP supports various operating systems:
32-bit versions of Microsoft Windows 64-bit versions of Microsoft Windows Microsoft Windows CE Linux32 Linux64 ARM Linux
Inside ippCV, the following computer vision functions are supported:
Image pyramid (resample and blur) Filters: Laplace, Sobel, Scharr, Erode, Dilate Motion gradient Flood fill Canny edge detection Snakes Optical flow
Of course, there are many other non-computer vision functions supported in 2500 functions that make up IPP. IPP includes support for image processing, compression, signal processing, audio and video codecs, matrix operations, and speech recognition.
Two other libraries that serve as higher level wrappers to IPP are of interest: the signal processing library and the image processing library [3]. OpenCV was designed to work well with both IPP and these libraries. The functions covered by the signal processing and image processing libraries follow.
Signal Processing Functions: Data manipulation Windowing Conversion Arithmetic and logical operations Filters and transforms Statistics Audio Signal generation Image Processing Functions: Data manipulation Arithmetic and logical functions Copy and conversions Geometric operations (scale, rotate, warp) Color conversion Alpha composite Gamma correction General and specific filters: FFT, DCT, wavelet transform
A final separate library to be aware of is the Math Kernel Library (MKL), which contains the full set of linear algebra and matrix manipulation routines.