Chapter 8. Content-Based Image Retrieval: An Overview
Theo Gevers and Arnold W. M. Smeulders
In this chapter, we present an overview on the theory, techniques, and applications of content-based image retrieval. We choose patterns of use, image domains, and computation as the pivotal building blocks of our survey. A graphical overview of the content-based image retrieval scheme is given in Figure 8.1. Derived from this scheme, we follow the data as they flow through the computational process (see Figure 8.3), with the conventions indicated in Figure 8.2. In all of this chapter, we follow the review in [155] closely.
Figure 8.1. Overview of the basic concepts of the content-based image retrieval scheme considered in this chapter. First, features are extracted from the images in the database, then are stored and indexed. This is done offline. The online image retrieval process consists of a query example image from which image features are extracted. These image features are used to find the images in the database that are most similar. Then, a candidate list of most similar images is shown to the user. From the user feedback, the query is optimized and used as a new query in an iterative manner.
Figure 8.3. Basic algorithmic components of query by pictorial example captured in a dataflow scheme while using the conventions of Figure 8.2.
Figure 8.2. Data flow and symbol conventions used in this chapter. Different styles of arrows indicate different data structures.
We focus on still images and leave video retrieval as a separate topic. Video retrieval could be considered a broader topic than image retrieval, as video is more than a set of isolated images. However, video retrieval could also be considered simpler than image retrieval, since, in addition to pictorial information, video contains supplementary information such as motion, spatial constraints, and time constraints (e.g., video discloses its objects more easily, as many points corresponding to one object move together and are spatially coherent in time). In still pictures, the user's narrative expression of intention is in image selection, object description, and composition. Video, in addition, has the linear timeline as an important information cue to assist the narrative structure.