i
i
i
i
i
i
i
i
Chapter 10
Image-Based Effects
“Landscape painting is really just a box of air with little marks
in it telling you how far back in that air things are.”
—Lennart Anderson
Modeling surfaces with polygons is often the most straightforward way to
approach the problem of portraying objects in a scene. Polygons are good
only up to a point, however. Image-based rendering (IBR) has become a
paradigm of its own. As its name proclaims, images are the primary data
used for this type of rendering. A great advantage of representing an object
with an image is that the rendering cost is proportional to the number of
pixels rendered, and not to, say, the number of vertices in a geometrical
model. So, one use of image-based rendering is as a more efficient way
to render models. However, IBR techniques have a much wider use than
this. Many objects, such as clouds and fur, are difficult to represent with
polygons. Layered semitransparent images can be used to display such
complex surfaces.
In this chapter, image-based rendering is first compared and contrasted
with traditional polygon rendering, and an overview of algorithms pre-
sented. Commonly used techniques such as sprites, billboards, particles,
and impostors are described, along with more experimental methods. Opti-
mizations are described for when the camera and scene are mostly
static.
At the same time, there is more involved in making an image than
simply portraying objects. We may wish to post-process synthesized data
in a wide variety of ways. Image processing techniques are explained for
performing tone mapping, high dynamic range effects, motion blur, depth
of field, and other phenomena. The chapter ends with two related top-
ics, atmospheric effects and using three-dimensional volumetric data for
modeling objects.
439
i
i
i
i
i
i
i
i
440 10. Image-Based Effects
Figure 10.1. The rendering spectrum. (After Lengyel [763].)
10.1 The Rendering Spectrum
The goal of rendering is to portray an object on the screen; how we at-
tain that goal is our choice. There is no single correct way to render a
scene. Each rendering method is an approximation of reality, at least if
photorealism is the goal.
Polygons have the advantage of representing the object in a reasonable
fashion from any view. As the camera moves, the representation of the
object does not have to change. However, to improve quality, we may wish
to substitute a more highly detailed model as the viewer gets closer to the
object. Conversely, we may wish to use a simplified form of the model
if it is off in the distance. These are called level of detail techniques (see
Section 14.7). Their main purpose is to make the scene display faster.
Other rendering and modeling techniques can come into play as an
object recedes from the viewer. Speed can be gained by using images
instead of polygons to represent the object. It is often less expensive to
represent an object with a single image that can be sent quickly to the
screen. One way to represent the continuum of rendering techniques comes
from Lengyel [763] and is shown in Figure 10.1. We will first work our way
from the left of the spectrum back down to the more familiar territory on
the right.
10.2 Fixed-View Effects
For complex geometry and shading models, it can be expensive to rerender
an entire scene at interactive rates. Various forms of acceleration can be
performed by limiting the viewer’s ability to move. The most restrictive
situation is one where the camera is fixed in position and orientation, i.e.,
does not move at all. Under such circumstances, much rendering can be
done just once.
For example, imagine a pasture with a fence as the static scene, with
a horse moving through it. The pasture and fence are rendered once and
i
i
i
i
i
i
i
i
10.2. Fixed-View Effects 441
the color and Z-buffers stored away. Each frame, these buffers are copied
over to the displayable color and Z-buffer. The horse itself is then all that
needs to be rendered to obtain the final image. If the horse is behind the
fence, the z-depth values stored and copied will obscure the horse. Note
that under this scenario, the horse cannot cast a shadow, since the scene is
unchanging. Further elaboration can be performed, e.g., the area of effect
of the horse’s shadow could be determined, and then only this small area of
the static scene would need to be rerendered atop the stored buffers. The
key point is that there are no limits on when or how the color gets set in
an image to be displayed. For a fixed view, much time can be saved by
converting a complex geometric model into a simple set of buffers that can
be reused for a number of frames.
It is common in computer-aided design (CAD) applications that all
modeled objects are static and the view does not change while the user
performs various operations. Once the user has moved to a desired view, the
color and Z-buffers can be stored for immediate reuse, with user interface
and highlighted elements then drawn per frame. This allows the user to
rapidly annotate, measure, or otherwise interact with a complex static
model. By storing additional information in G-buffers, similar to deferred
shading (Section 7.9.2), other operations can be performed. For example, a
three-dimensional paint program can be implemented by also storing object
IDs, normals, and texture coordinates for a given view and converting the
user’s interactions into changes to the textures themselves.
A concept related to the static scene is golden thread or adaptive refine-
ment
1
rendering [83, 1044]. The idea is that while the viewer and scene are
not moving, the computer can produce a better and better image as time
goes on. Objects in the scene can be made to look more realistic. Such
higher-quality renderings can be swapped in abruptly or blended in over
a series of frames. This technique is particularly useful in CAD and visu-
alization applications. There are many different types of refinement that
can be done. One possibility is to use an accumulation buffer to do anti-
aliasing (see Section 5.6.2) and show various accumulated images along the
way [894]. Another is to perform slower per-pixel shading (e.g., ray trac-
ing, ambient occlusion, radiosity) off screen and then fade in this improved
image.
Some applications take the idea of a fixed view and static geometry a
step further in order to allow interactive editing of lighting within a film-
quality image. The idea is that the user chooses a view in a scene, then uses
this data for offline processing, which in turn produces a representation of
the scene as a set of G-buffers or more elaborate structures. Such techniques
1
This idea is also known as “progressive refinement.” We reserve this term for the
radiosity algorithm by the same name.
i
i
i
i
i
i
i
i
442 10. Image-Based Effects
are used in digital film production packages, such as NVIDIA’s Sorbetto,
to allow real-time relighting of highly detailed scenes.
Guenter et al. [466] were one of the first groups to explore this area.
They describe a system that allows editing of local lighting model param-
eters. It automatically optimizes computations for the current parameter,
giving an increase in rendering speed of as much as 95 times. Gershbein
et al. [391] implement a package that includes BRDF storage and shadow
occlusion. Pellacini et al. [997, 998] describe a modern cinematic relight-
ing system that is able to also store indirect illumination from multiple
bounces. Ragan-Kelley et al. [1044] present their own package, which keeps
shading samples separate from final pixels. This approach allows them to
perform motion blur, transparency effects, and antialiasing. They also use
adaptive refinement to improve image quality over time. These techniques
closely resemble those used in deferred shading approaches (described in
Section 7.9.2). The primary difference is that here, the techniques are
used to amortize the cost of expensive rendering over multiple frames, and
deferred shading uses them to accelerate rendering within a frame.
An important principle, introduced in Section 7.1, is that for a given
viewpoint and direction there is an incoming radiance. It does not matter
how this radiance is computed or at what distance from the eye it is gener-
Figure 10.2. A panorama of the Mission Dolores, used by QuickTime VR to display a
wide range of views, with three views below, generated from it. Note how the views
themselves are undistorted. (Images courtesy of Ken Turkowski.)
i
i
i
i
i
i
i
i
10.3. Skyboxes 443
ated; the eye does not detect distance, only color. Capturing the radiance
in a given direction in the real world can be done by simply
2
snapping a
photograph. In the Quicktime VR system [172] a 360-degree panoramic
image surrounds the (unmoving) viewer as a cylindrical image. The image
is usually created from a set of photographs stitched together. As the cam-
era’s orientation changes, the proper part of the image is retrieved, warped,
and displayed. Though limited to a single location, this technique has an
immersive quality compared to a fixed view, because the viewer’s head can
turn and tilt. See Figure 10.2. Kim and Hahn [655] and Nielsen [934] dis-
cuss efficient use of cylindrical panoramas on the GPU. It is also possible
to store the distance or other values of each texel’s contents and so allow
dynamic objects to interact with the environment.
10.3 Skyboxes
Pick up this book and look just past the left or right edge toward what is
beyond it. Look with just your right eye, then your left. The shift in the
book’s edge compared to what is behind it is called the parallax. This effect
is significant for nearby objects, helping us to perceive relative depths as
we move. However, for some group of objects sufficiently far away from
the viewer, there is barely any parallax effect when the viewer moves. In
other words, a distant mountain itself does not normally look appreciably
different if you move a meter, or even a thousand meters. It may be blocked
from view by nearby objects as you move, but take away those objects and
the mountain itself looks the same.
An environment map represents the incoming radiance for a local vol-
ume of space. While such maps are typically used for simulating reflections,
they can also be used directly to represent distant objects in the surround-
ing environment. Any view-independent environment map representation
can be used for this purpose; cubic maps are the most common. The
environment map is placed on a mesh centered around the viewer, and
large enough to encompass the rest of the objects in the scene. The exact
shape of the mesh does not matter; a cube is often used. This mesh is
called a skybox. An example is shown in Figure 9.49 on page 397. The
skybox shown in that figure is more like an unconstrained version of a
Quicktime VR panorama. The viewer can look in any direction, but any
movement would destroy the illusion for this scene, as we would detect no
parallax. Environment maps can often contain objects that are relatively
close to the reflective object. The effect works because we normally do not
2
“Simply” is a relative term. Factors such as noise, glare, motion, and depth of field
can make the process of capturing a high-quality photograph a challenge. Computer
graphics has the advantage of full control over these various areas.
..................Content has been hidden....................
You can't read the all page of ebook, please click here login for view all page.