i
i
i
i
i
i
i
i
474 10. Image-Based Effects
10.10 Color Correction
Color correction
8
is the process of taking an existing image and using some
function to convert each color to some other color (or not converting at
all). There are many reasons to color correct images or film, e.g., to mimic
the appearance of a specific kind of film stock, provide a coherent look
among elements, or to portray a particular mood or style. Color correction
typically consists of taking a single pixel’s RGB value as the input and
applying an algorithm to it to produce a new RGB. Another use is to
accelerate decoding video from, for example, the YUV color space to RGB.
More complex functions that rely on screen position or neighboring pixels
are possible [1148], but most operators use the color of each pixel as the
sole input. Some color correction functions treat each color separately,
i.e., the value of the red input is the only variable that affects the red
output channel. Others use all three channels as inputs that can affect each
output. The RGB to luminance conversion, Equation 7.11 on page 215, is
an example of the latter: All three inputs affect each output channel. In
this case, the output value is the same for all three outputs.
For a simple conversion such as luminance, a pixel shader program can
be applied with a screen-filling quadrilateral and the equation evaluated
directly. Simple formulae can perform basic operations such as scaling the
brightness or modifying saturation [476]. However, color correction func-
tions can be anything: complex mathematical functions or user-generated
tweaks and modifications. In addition, these functions can be mixed or
used multiple times. The final correction function can therefore be arbi-
trarily complex and so be expensive to apply in an interactive setting. On
the GPU, the normal solution for evaluating complex functions is to use a
look-up table (LUT). For a color correction function that has only one input
channel, the table can be one-dimensional. The input acts as the index into
the array, and the output value for that input is stored in the array. In
this way, any arbitrarily complex function for a single channel is reduced
to a constant-time lookup. Implementing this on the GPU is a matter of
storing the array in a texture. Given, say, 256 possible values for the input
color, a 1 × 256 texture is created to hold the output values. A texture
read is then used to access this array to perform the conversion [94].
When a color conversion uses three inputs, a three-dimensional LUT
of size 256
3
(more than 16 million texels) is usually out of the question.
If the color conversion function does not change rapidly between neigh-
boring values, a much smaller LUT array can be used, e.g., a cube of say
32
3
values. The resolution of the volume depends on the transform’s lin-
earity. The GPU’s texture sampling functionality can be used to perform
8
In film production, often called color grading or color timing.
i
i
i
i
i
i
i
i
10.11. Tone Mapping 475
Figure 10.24. Color correction is performed on the scene on the left to create the effect
on the right. By dimming and desaturating the colors, a nighttime effect is produced.
A volume texture is used to transform the colors of the original image [881]. (Images
from “Day of Defeat: Source” courtesy of Valve Corp.)
trilinear interpolation among the neighboring values. Usually such tables
will need to have higher precision than eight bits to avoid banding arti-
facts. An example from Valve’s color correction system [881] is shown in
Figure 10.24. Selan’s article [1148] provides a good description of color
conversion techniques for the GPU.
10.11 Tone Mapping
A computer screen has a certain useful luminance range, while a real image
has a potentially huge luminance range. Tone mapping (also called tone
reproduction) is the process of fitting a wide range of illumination levels
to within the screen’s limited gamut [286, 1274]. The fact that the eye
takes time to adjust to rapidly changing lighting conditions, called light
adaptation, can be simulated in real-time rendering by artificially changing
the perception of light. For example, for a person coming out of a tunnel
in a virtual world, the scene could be made to appear bright and washed
out for a second and then fade to normal light and color levels. This
would mimic the perceived feeling of coming out of a real tunnel into the
light [990].
There are many other cases where tone reproduction and the eye’s re-
sponse play a part. As a demonstration, take a CD jewel case and look
down on it. Aim it to reflect a fluorescent light source (i.e., a light source
that is not painful to view directly). By the Fresnel effect, the reflection
i
i
i
i
i
i
i
i
476 10. Image-Based Effects
from the CD case is only about 4% the power of the light itself. Our eyes
rapidly adjust when shifting from the CD to the light, and so we barely
notice this wide disparity.
Older interactive computer rendering systems could not handle color
values outside the 0.0to1.0 range. This limited the inputs, as well as the
outputs, of the shading equation to this range. The limitation on input
values was the more restrictive of the two. The fact that any quantity
could not exceed 1 meant that any product of two quantities was always
darker than either. For example, diffuse lighting could darken the diffuse
color texture but never brighten it. These limitations meant that all quan-
tities had to be carefully tweaked to a narrow brightness range, leading
to washed-out scenes with low contrast and little detail in shadowed re-
gions. This not only reduced realism, but also made the resulting images
less visually appealing.
Modern GPUs can handle almost arbitrarily high shading values, al-
lowing for realistic lighting and high contrast images. Since output devices
are still limited in their supported range and precision (8 bits per channel
is typical)
9
some form of tone mapping needs to be done on the shaded
values. This mapping can be performed on the fly (as the objects are ren-
dered) or in a separate full-screen pass. An advantage of tone mapping on
the fly is that low-precision color buffers can be used. However, there are
two disadvantages to on-the-fly tone mapping. One is that in scenes with
high overdraw, the tone mapping step will be performed multiple times per
pixel, which is particularly troublesome if the operator is costly. The other
drawback has to do with alpha-blended objects. Tone mapping should
properly be performed after blending, and doing it on the fly may result in
objectionable artifacts. Performing tone mapping in a separate pass solves
these issues, but does require a high-precision color buffer. Such color
buffers use more memory bandwidth, an increasingly precious resource.
The simplest “tone mapping operator” is to simply clamp the output
values to the displayable range. In many applications where the lighting
and camera are predictable, this can work surprisingly well. An extra
degree of control can be added via an exposure setting that scales the
output colors before they are clamped. Such controls can be set by the
application designers or artists for best visual effect.
In applications where the lighting is not predictable, adaptive tone map-
ping techniques that react to the overall illumination in the scene can pro-
duce better results. For first-person games in particular, such mimicry of
the human visual system can lead to a more immersive experience. The
simplest adaptive tone mapping operator is maximum to white [1010]. The
9
Displays with a greater effective contrast and brightness range are available. This
subject is touched upon in Section 18.1.1.
i
i
i
i
i
i
i
i
10.11. Tone Mapping 477
largest luminance value found is used to scale the results to a displayable
range. One major drawback of this method is that a single exceptionally
bright pixel in the scene will make the rest of the results all scale to be
extremely dark.
What our eyes most notice are differences in contrast in the area cur-
rently being viewed. Tone mapping operators use each pixel’s computed
luminance value to determine the RGB displayed. Such operators can be
global or local in nature. A global algorithm, such as “maximum to white,”
applies the same mapping to all parts of the image. A local algorithm an-
alyzes each pixel with respect to its neighbors. If a neighbor is extremely
different in luminance value, its effect on the pixel’s mapping is minimized,
as only those pixels that are close in luminance are the ones most notice-
able for maintaining contrast. Local tone mapping operators can be quite
effective, but are not often implemented in interactive applications. Good-
night et al. [426] present a system that implements both global and local
tone operators using pixel shaders.
Global tone mapping operators are less costly in general. There are
effectively two steps: Sample the computed image to determine luminance
levels, then use this information to remap the image as desired. For exam-
ple, the “maximum to white” method examines every pixel for the largest
luminance, then scales all pixels by this maximum value. Running through
all pixels in an image can be costly. One method is to perform a number of
passes. The first pass renders a screen-filling quad to a smaller image, us-
ing numerous texture samples to downsize and average the luminance. The
succeeding passes sample this small image to reduce it further, eventually
to a single pixel holding the average luminance [152, 1206]. A variant is
to make a direct rendering of a smaller version of the scene, i.e., a quarter
size or smaller, then average this. This smaller image is useful for post
processing to produce a bloom or glow effect for the brighter pixels in the
image, as discussed in Section 10.12. Other sampling schemes are possible,
such as sampling over fewer pixels or sampling using some conversion other
than the luminance equation [261].
Another approach is to create a histogram of the rendered result’s lu-
minance and use it to create a mapping. A histogram is an array of bins
and is commonly represented as a bar graph, with each bar’s height show-
ing the number of pixels of a given luminance value or range of values.
Histograms can also be created for individual RGB channels. One simple
mapping is histogram equalization [422], which remaps the histogram so
that each display luminance level has a number of bins assigned to it. The
goal is to have each display level have approximately the same number of
pixels associated with it. The effect is to increase contrast among the more
common luminance levels. One problem with this approach is that contrast
can be over-exaggerated. Scheuermann and Hensley [159, 1122] provide an
i
i
i
i
i
i
i
i
478 10. Image-Based Effects
efficient method of generating histograms with the GPU by using vertex
texture fetches or rendering to a vertex buffer. They also give an overview
of previous work and alternate methods, as well as implementation details
for histogram equalization and tone mapping operators that avoid contrast
exaggeration.
Valve [881] uses GPU occlusion queries to count the pixels that fall in
a given range of luminance values, building up the luminance histogram
over several frames (time averaging is used to smooth out high-frequency
variations in the results). Systems such as CUDA, which enable writing
from the GPU to different memory locations, can enable the generation of
luminance histograms on the fly, without additional passes.
If the range of input values from the image is relatively low, i.e., it
does not encompass many magnitudes of difference, a simple averaging
and rescaling can work fine to map the image to a displayable range [1206].
Normally, taking the straight average of the luminance values for high dy-
namic range images is inadequate for reasonable tone mapping operations,
since a few extremely large values can dominate the average computed. As
summarized by Reinhard et al. [1058], when summing up the pixel values,
it is usually better to take the logarithm of the luminance and average these
values, then convert back, i.e.,
L
w
=exp
1
N
x,y
log(δ + L
w
(x, y))
. (10.8)
L
w
is the log-average luminance and L
w
(x, y) is the luminance at pixel
(x, y). The δ is some tiny value, e.g., 0.0001, just so the logarithm of 0
(undefined) is never attempted for black pixels.
The effect of light adaptation can be applied at this point, if desired.
One method is to use the previous frame’s log-average luminance along
with this frame’s [261]:
L
w
= L
prev
w
+ c(L
w
− L
prev
w
). (10.9)
Setting c to, say, 0.04 has the effect of taking 4% of the difference between
the current luminance and the previous luminance and adapting by this
much for the frame.
10
The resulting luminance L
w
is then used as the
previous luminance in the next frame. This is a simple hack with little
physical basis; other equations are certainly possible [159, 1206]. Such
blending based on previous results can also have the desirable effect of
damping out any statistical noise from one frame to the next. The slowly
changing nature can, in turn, be used to reduce the number of samples
10
This equation assumes a constant frame rate. In practice, a term such as 1 −
pow(c, t/30) should be used in place of c,wheret is time in seconds and 30 is the fps.
..................Content has been hidden....................
You can't read the all page of ebook, please click here login for view all page.