7.1. Introduction

Computer vision has changed the way movies are made. Sure, computer graphics gets all the press, but computer vision techniques have made a significant impact on the way films with visual effects are envisioned, planned, and executed. This chapter examines the state of the practice of computer vision techniques in visual effects. We also examine the changes computer vision has brought to the industry, the new capabilities directors and visual effects creators have available, and what is desired in the future.

Let us examine the concept of visual effects in terms of computer vision. The main task of visual effects is to manipulate or add things to an image. Artists are very skilled, but the more information they have about the original image and the more tools the computer presents to them, the more effective they can be.

Computer vision has made a huge impact in the field of visual effects over the last decade. In the early 1990s, we saw the shift away from using physical models and photographic techniques to create and add fantastic things to an image. It has became commonplace to scan each frame of film into the computer and composite computer-generated elements with the filmed background image. Digitizing the filmed image is so common now that whole films are digitized, manipulated in the digital realm, and filmed out to a negative for printing. (Oh Brother, Where Art Thou? is a good example of this. The color of each frame of the film was digitally adjusted to give the film its "look.") And, of course, digital cameras are starting to make inroads into Hollywood. Star Wars, Attack of the Clones was shot completely with a digital camera.

Once a sequence of frames is digitized, it becomes possible to apply standard computer vision algorithms to the digitized sequence in order to extract as much information as possible from the image. Structure from motion, feature tracking, and optical flow are just some of the techniques that we examine in relation to filmmaking.