Noise, sharpness, resolution and information
INTRODUCTION
Imaging systems transmit information and can be analysed using the procedures developed for other types of communication systems. In Chapter 7, the analysis of frequency response was introduced and developed as one of the most important of these procedures. In the context of images, this gives the means of assessing the progress of information carrying spatial signals through an imaging chain.
In common with other communication systems, this signal transfer proceeds against a background of noise. Indeed, the true effectiveness of a system for transmitting information can only be measured in terms of the signal-to-noise ratio. Most objective image quality metrics (see Chapter 19) involve some form of signal-to-noise component. In this chapter image noise and sharpness are further examined, and the ways in which these limit the usefulness of images and imaging systems as carriers of information. Fundamental measures, such as detective quantum efficiency (DQE) and information capacity, can then be defined without reference to the internal technology involved in the imaging system. The definitions use signal-to-noise (S/N) ratios, which in turn depend on the system modulation transfer function (MTF) and output noise power. These can all be measured from the output image. When applied to specific imaging systems and processes, these fundamental measures offer additional scope for modelling, which in turn allows a greater understanding of the limits of the various technologies.
IMAGE NOISE
Image noise is essentially the unwanted fluctuations of light intensity over the area of the image. It is a quantity that usually varies over the image plane in some random way, although there are examples of image corruption from structured patterns such as sinusoidal interference patterns (sometimes referred to as coherent noise) and fixed pattern noise associated with charge-coupled devices (CCDs).
Figure 24.1 shows an image with and without noise. We usually consider the noise to be a random pattern added to (or superimposed upon) the true image signal. Although there is an intrinsic unpredictability, a random process can be defined statistically, for example by its mean value, variance or probability distribution. The spatial properties can be summarized by the autocorrelation function and the power spectrum.
Causes of noise in an image
1. Random partitioning of exposing light.
2. If an image is converted from a light intensity of exposure distribution to an electrical form, there will be photoelectric noise.
3. Electronic noise will occur in any system with electronic components.
4. Quantization noise occurs at the digitization stage in digital systems.
5. Poisson exposure noise means there is a randomness of photons in a nominally uniform exposure distribution.
The first of these is the cause of image noise in photographic systems where images exhibit noise due to the random grain structure of photographic negatives. The second and third cases are the main cause of image noise at the output of a CCD or complementary metal oxide semiconductor (CMOS) sensor. Case 4 occurs to a greater or lesser extent in all digital systems where a brightness value is quantized (or binned) into one of a finite number of values, as described in Chapter 9. Poisson exposure noise is present in all images. It arises because the photons of light captured by the imaging system arrive randomly in space and time. The relative magnitude, and hence significance, of quantum noise will depend on the system and its use.
Figure 24.1 (a) An image without perceptible noise. (b) The same image with increased noise.
The best-known examples of noisy images occur with photographic systems. Anyone who has attempted an enlargement of more than about × 10 from a photographic negative will be familiar with the appearance of a grainy structure over the image. Photographic noise is well studied and the procedures for quantifying it are straightforward. A discussion of photographic noise is first presented with reference to the traditional black-and-white negative–positive process.
PHOTOGRAPHIC NOISE
Graininess
The individual crystals of a photographic emulsion are too small to be seen by the unaided eye, even the largest being only about 2 mm in diameter. A magnification of about ×50 is needed to reveal their structure. A grainy pattern can, however, usually be detected in photographic negatives at a much lower magnification, sometimes as low as only three or four diameters. There are two main reasons for this:
1. Because the grains are distributed at random, in depth as well as over an area, they appear to be clumped. This apparent clumping forms a random irregular pattern on a much larger scale than with the individual grain size.
2. Not only may the grains appear to be clumped because of the way in which they are distributed in the emulsion, but they may actually be clumped together (even in physical contact) as a result either of manufacture or of some processing operation.
The sensation of non-uniformity in the image produced in the consciousness of the observer when the image is viewed is termed graininess. It is a subjective quantity and must be measured using an appropriate psychophysical technique, such as the blending magnification procedure. The sample is viewed at various degrees of magnification and the observer selects the magnification at which the grainy structure just becomes (or ceases to be) visible. The reciprocal of this ‘blending magnification factor’ is one measure of graininess. Graininess varies with the mean density of the sample. For constant sample illumination, the relationship has the form illustrated in Figure 24.2. The maximum value, corresponding to a density level of about 0.3, is said to result from the fact that approximately half of the field is occupied by opaque silver grains and half is clear, as might intuitively be expected. At higher densities, the image area is more and more covered by agglomerations of grains and, because of the reduced acuity of the eye at the lower values of field luminance produced, graininess decreases.
We have so far referred only to the graininess of the negative material. The feature of interest in pictorial photography is likely to be the quality of a positive print made from that negative. If a moderate degree of enlargement is employed the grainy structure of the negative becomes visible in the print (the graininess of the paper emulsion itself is not visible because it is not enlarged). Prints in which the graininess of the negative is plainly visible are in general unacceptable in quality. Although measures of negative graininess have successfully been used as an index to the quality of prints made from the negative materials, the graininess of a print is not simply related to that of the negative, because of the reversal of tones involved. Despite the fact that the measured graininess of a negative sample decreases with increasing density beyond a density of about 0.3, the fluctuations of micro-density across the sample in general increase. This follows from the increase in grain number involved. As these fluctuations are responsible for the resulting print graininess, and as the denser parts of the negative correspond to the lighter parts of the print, we can expect relatively high print graininess in areas of light and middle tone (the graininess of the highlight area is minimal because of the low contrast of the print material in this tonal region). This result supplements the response introduced by variation of the observer’s visual acuity with print luminance. Figure 24.3 shows the graininess relationship between negative and positive samples.
It is clear that print graininess is primarily a function of the fluctuations of microdensity, or granularity, of the negative. The graininess of the negative itself is of little relevance. Hence, when a considerable degree of enlargement is required, it is desirable to keep the granularity of the negative to a minimum.
Figure 24.2 Graininessedensity curves for a typical medium-speed film.
Figure 24.3 Graininess in negative and print. Left: enlargement of negative. Right: corresponding areas of print.
Factors affecting the graininess of prints
1. The granularity of the negative. As this increases, the graininess of the print increases. This is the most important single factor and is discussed in detail in the next section.
2. Degree of enlargement of the negative.
3. The optical system employed for printing and the contrast grade of paper chosen.
4. The sharpness of the negative. The sharper the image on the film, the greater the detail in the photograph and the less noticeable the graininess. It can be noted here that graininess is usually most apparent in large and uniform areas of middle tone where the eye tends to search for detail.
5. The conditions of viewing the print.
Granularity
This is defined as the objective measure of the inhomogeneity of the photographic image, and is determined from the spatial variation of density recorded with a microdensitometer (a densitometer with a very small aperture). A typical granularity trace is shown in Figure 24.4. In general, the distribution of a large number of readings from such a trace (taken at intervals greater than the aperture diameter) is approximately normal, so the standard deviation, s (sigma), of the density deviations can be used to describe fully the amplitude characteristics of the granularity. The standard deviation varies with the aperture size (area A) used. For materials exposed to light, the relationship 
(a constant, termed the Selwyn granularity coefficient – see later) holds well and the parameter S can be used as a measure of granularity.
Figure 24.4 Granularity trace for a medium-speed film, using a scanning aperture of 50 mm.
Factors affecting negative granularity
The following are the most important factors affecting the granularity of negatives:
1. Original emulsion employed. This is the most important single factor. A large average crystal size (i.e. a fast film) is generally associated with high granularity. A small average crystal size (i.e. a slow film) yields low granularity.
2. The developing solution employed. By using fine-grained developers it is possible to obtain an image in which the variation of density over microscopic areas is somewhat reduced.
3. The degree of development. Since granularity is the result of variations in density over small areas, its magnitude is greater in an image of high contrast than in one of low contrast. Nevertheless, although very soft negatives have lower granularity than equivalent negatives of normal contrast, they require harder papers to print on, and final prints from such negatives usually exhibit graininess similar to that of prints made from negatives of normal contrast.
4. The exposure level, i.e. the density level. In general, granularity increases with density level. Typical results are shown in Figure 24.5. This result leads to the conclusion that overexposed negatives yield prints of high granularity.
The first two factors are closely related, for just as use of a fast emulsion is usually accompanied by an increase in granularity, so use of a fine-grained developer often leads to some loss of emulsion speed. Consequently, it may be found that use of a fast film with a fine-grained developer offers no advantage in effective speed or granularity over a slower, finer-grained film developed in a normal developer. In fact, in such a case, the film of finer grain will yield a better modulation transfer function and higher resolving power, and in consequence is to be preferred.
Figure 24.5 Granularity as a function of density for high-(a), medium- (b) and slow-speed (c) films.
VARIATION OF GRANULARITY WITH DENSITY
Figure 24.2 shows the non-linear relation between graininess and image density for a typical silver image, while Figure 24.5 relates Selwyn granularity to density for monochrome films of differing speeds. In each case the images are silver, developed in a standard monochrome developer. In some commercial products, the monochrome emulsion contains colour couplers to yield a dye image in a colour developer, and the silver image is bleached away in the processing, leaving a dye image. This behaves differently from a silver image and is illustrated in Figure 24.6a. and b. In Figure 24.6a. the silver-image granularity is compared with that of the dye image, also generated by the chromogenic, dye-forming developer used. Both sets of granularity data are plotted against visual density. The dye image gives much lower granularity at high than at low densities, while the silver image possesses a slightly diminished granularity at high density. This product is normally treated in a combined bleach and fix bath after development, leaving only the dye image and giving the dashed curve in Figure 24.6. On the other hand, if the film is processed as a conventional monochrome film, a silver image alone is obtained and the granularity behaviour is more conventional (Figure 24.6b). This behaviour is important: conventional negative film gives grainier results on overexposure while chromogenic negative gives less grainy images. It means that chromogenic negative film can be overexposed considerably, i.e. rated at a lower ISO speed, giving the lower granularity associated with conventional negative films of that lower speed. Carefully formulated chromogenic negative film, using special colour couplers, can yield increased sharpness as well as decreased graininess, when rated at lower ISO speeds. The image density, however, will be higher than usual.
QUANTIFYING IMAGE NOISE
Methods for quantifying image noise are presented in the following sections in terms of the photographic imaging process, where noise fluctuations are measured in density units (denoted D). The methods can be applied quite generally to other imaging systems where the fluctuations might, for instance, be luminance or digital values.
Although we are dealing with two-dimensional images it is usual to confine noise analysis to one spatial dimension. This simplifies the methods and is entirely adequate for most forms of noise. We begin by assuming we have a noise waveform D(x) – the variation of output density in one direction across a nominally uniform sample image.
Figure 24.6 Granularity of chromogenic negative film, as a function of visual density, developed in: (a) colour developer for the recommended time and fixed, but with silver unbleached, silver and dye images being separately measured; (b) black-and-white developer at times giving the γ values shown.
© Geoffrey Attridge
Variance
This is the mean square noise, or the total noise power, and is useful for summarizing the amplitude variation of the (Gaussian) random function. It is defined as:
where ∆ D(x) is the deviation of D(x) from the mean density.
Estimating σ2
The expression for variance, σ2, is replaced by a discrete version for the purpose of calculation:
∆Di is the ith measured value of the deviation. N values are recorded (where N is ‘large’, for example 1000) using a measuring aperture of area A. Note that this is simply the formula for sample variance found in textbooks on statistics.
Selwyn’s law
Studies of photographic granularity show that, provided the scanning aperture area A is ‘large’ compared with the grain clump size, the product of σ2A with A is a constant for a given sample of noise. This is known as Selwyn’s law and means that σ2A is independent of the value of A. Hence σ2A can be used as a measure of image noise (granularity). Other measures include 
. The latter expression is known as Selwyn granularity. This is often written as:
where S is the measure of granularity.
Although σ or σ2 can be used to describe the amplitude characteristics of photographic noise, a much more informative approach to noise analysis is possible using methods routinely used in communications theory. As well as amplitude information, these methods supply information about the spatial structure of the noise.
The autocorrelation function
Instead of evaluating the mean square density deviation of a noise trace, the mean of the product of density deviations at positions separated by a distance τ (tau) is measured for various values of τ. The result, plotted as a function of τ, is commonly known as the autocorrelation function C(τ). It is defined mathematically as:
It includes a measure of the mean square density deviation (τ = 0) but, equally important, contains information about the spatial structure of the granularity trace. Strictly, Eqn 24.32 gives the autocovariance of the signal and should be normalized by the variance to yield the autocorrelation function. The imaging field commonly uses the autocorrelation function and we will keep with this convention, but readers should be aware of the difference.
The importance of the function is illustrated in Figure 24.7, where a scan across two different (greatly enlarged) photographic images is shown. The granularity traces, produced using a long thin slit, have a characteristic that is clearly related to the average grain clump size in the image. The traces are random, however, so to extract this information the autocorrelation function is used. The two autocorrelation functions shown summarize the structure of the two traces, and can therefore be used to describe the underlying random structure of the image. Larger grain structures will generally produce autocorrelation functions that extend further, whereas finer grain will cause the autocorrelation function to fall more rapidly.
The noise power spectrum
The autocorrelation function is related mathematically (via the Fourier transform) to another function of great importance in noise analysis, the Wiener or power spectrum. This function can be obtained from a frequency analysis of the original granularity trace, and expresses the noise characteristics in terms of spatial frequency components.
A practical definition of the noise power spectrum is:
It is essentially the squared modulus of the Fourier transform (see Chapter 7) of the density deviations, divided by the range of integration, x. Formally, the noise power spectrum represents the noise power per unit bandwidth plotted against spatial frequency. It is important to note that in practice Eqn 24.5 does not converge as longer data lengths are used. This is because the bandwidth of each measurement point becomes smaller and statistical error associated with measuring a random process (noise) remains. To overcome this problem data is generally divided into smaller lengths, Eqn 24.5 applied and the ensemble average of the results taken to yield an estimate of the noise power spectrum (see later comments in the section ‘Direct Fourier transformation of noise samples’).
Figure 24.7 Autocorrelation functions and noise power (Wiener) spectra for two different image structures: (a) coarse grain; (b) fine grain.
The power spectra of the two granularity traces are also shown in Figure 24.7. It will be noticed that the trend in shape is opposite, or reciprocal, to that of the autocorrelation functions. The fine-grained image has a fairly flat spectrum, extending to quite high spatial frequencies. This reflects the fact that the granularity trace contains fluctuations that vary rapidly, as compared with the coarse-grained sample in which the fluctuations are mainly low in frequency.
Relationships between noise measures
For a given noise trace the three measures discussed above are related as follows:
• the variance is the area under the noise power spectrum;
• the noise power spectrum is the Fourier transform of the autocorrelation function.
Although the noise power spectrum and the autocorrelation function are closely related (one may be readily calculated from the other), they have very distinct roles in image evaluation. The autocorrelation function relates well to the causes of image noise while the power spectrum is important in assessing its effects.
In all the above, the equivalent two-dimensional versions of the definitions and relationships exist for the more general case of two-dimensional image noise ‘fields’.
Practical considerations for the autocorrelation function and the noise power spectrum
In practice, image noise is sampled either intrinsically as in the case of digital systems or following some external scanning as in the evaluation of photographic images.
The expressions for the autocorrelation function and the power spectrum defined in the previous sections have discrete equivalents for the purpose of calculation. To use them, the sampling theorem must be obeyed for the results not to be distorted by aliasing.
The noise field (i.e. a two-dimensional image containing noise alone) is scanned and sampled using a long thin slit to produce a one-dimensional trace for analysis. In the case of photographic noise this is an optical scanning process and would typically use a slit of width 2–4 mm. The sampling interval dx must be selected to avoid aliasing (defined in Chapter 7), i.e.
where uc is the maximum significant frequency contained by the noise. In the photographic case uc is likely to be determined by the MTF of the scanning slit (see Chapter 7). This means that the sampling interval δx may need to be as small as 1–2 mm.
For a digital imaging system, the philosophy is a little different. Whatever the source of noise in the digital image (see sources of electronic noise later in the chapter), it can only contain frequencies up to the Nyquist frequency (defined by the pixel separation). To evaluate the noise in the x direction and up to the Nyquist frequency for this virtual image, columns of data are averaged to produce a one-dimensional noise trace. To deal with noise in the γ direction, rows of data are averaged.
The total number N of noise samples used must be large (e.g. 10,000) to obtain a reasonable degree of accuracy. In the photographic case a scan distance of 10–20 mm is required to achieve this. For a digital system, many frames containing noise will need to be evaluated.
Using the autocorrelation function
The first (n/2 + 1) points of the autocorrelation function (this includes s = 0) are calculated using:
where n is selected using the sampling theorem:
where δu is the required bandwidth of the measurement (i.e. the interval along the spatial frequency axis at which the noise power spectrum values are required).
A Fourier transform of the n values of the (symmetrical) autocorrelation function gives the noise power spectrum. To ensure the values of the spectrum correctly represent the power in the original two-dimensional image, the function should be re-scaled as follows:
• Determine the variance, σ2A, when the noise field is scanned with a ‘large’ aperture of area A (i.e. large compared to the above slit width). In the photographic case this might be an aperture of size 50× 50 mm2. In the case of a digital image it will be an equivalent simulation.
• Form the quantity Aσ2A. This is the required value of the noise power spectrum at the origin, i.e.
Direct Fourier transformation of noise samples
The N values of noise are Fourier transformed directly. The squared modulus of the result is the noise power spectrum. Due to Eqn 24.9, this will have an extremely narrow bandwidth.
In other words, the spectrum is determined at very close intervals along the spatial frequency axis. Because noise is non-deterministic (averaging successive samples does not converge to a ‘true’ noise trace), each computed point in the spectrum is subject to high error, which does not reduce as the noise sample is lengthened. To get a reasonable estimate of the spectrum, blocks of adjacent spectrum values are averaged to increase the effective bandwidth and reduce the error per point (this is known as band averaging). The resulting noise power spectrum should be scaled as described in the previous paragraph.
Figure 24.8 shows noise power spectra for developed single-layer coatings of silver halide/cyan coupler with increasing concentrations of coupler DIR. The noise power spectra were measured with a microdensitometer fitted with a red filter. The sampling interval δx was 3 μm, giving a Nyquist frequency of 167 μm−1. A slit of width 4 mm was used to supply optical low-pass filtration to avoid aliasing of unwanted high frequencies into the region of interest. The sample size N = 40,960 gives a measurement bandwidth δu = 8.14 ×10−1
Figure 24.8 Noise power spectra for developed single-layer coatings of silver halide/cyan coupler with increasing concentrations of coupler DIR (a–c).
Curves redrawn from Graves, H.M., Saunders, A.E. (1984). The noise-power spectrum of colour negative film with incorporated couplers. Paper presented at the RPS Symposium on Photographic and Electronic Image Quality, Cambridge, September
Blocks of approximately 600 points were averaged to give the final curves, which have an effective bandwidth of 5 × 10−3 mm−1. It should be noted that some aliasing is likely to have occurred.
If the noise values are uncorrelated, the noise power spectrum will be constant at all frequencies (‘white noise’). There is no point in evaluating the spectrum. All that is required is the value of Aσσ2A.
Signal-to-noise ratio
In communication theory, the signal-to-noise ratio (S/N) is the ratio of the information carrying signal to the noise that tends to obscure that signal. In the case of images, the signal is some measure of the amplitude, or contrast, of the wanted image structure. The noise is the equivalent measure of the unwanted, usually random, superimposed fluctuations.
The particular measures of signal and noise employed will depend on the type of image and the application. A simple example is obtained from the field of astronomical image recording. Figure 24.9. shows a star image in the form of a disc of area A mm2 superimposed on a noisy background. If we imagine scanning across the image with an aperture of the same size as the star image, we would produce a trace of the form shown in Figure 24.9.. The maximum deflection, recorded when the aperture just sits on top of the star image, is shown as the signal strength, DS. The standard deviation of the background fluctuations, σB, gives the corresponding measure of noise. The signal-to-noise ratio is defined as:
Figure 24.9 (a) Image of a star (circular disc). (b) Trace across image, recorded with a scanning aperture the same size as the star.
For more general images, particularly in digital form, a signal-to-noise ratio can be calculated by evaluating the total variance of the image σ2T and the variance σ2N of a nominally uniform portion of the image (the noise). The variance of the signal is then the difference σ2T σ2N(assuming the noise is uncorrelated with the signal). The signal-to-noise ratio is then:
A more rigorous treatment of signal-to-noise is afforded by utilizing the ideas of Fourier theory. The signal is considered to be a set of spatial frequencies determined by the MTF of the image-forming system. The noise is represented by the image noise power spectrum. The signal-to-noise ratio is then dependent on the MTF of the system ‘viewing’ the image. Where the viewing system is the human observer the noise power spectrum is often presented in the form of a modulation threshold curve. This displays the minimum modulation necessary for a sinusoid signal to be visible above the noise, as a function of spatial frequency. Expressions for perceived image quality can then be constructed.
ELECTRONIC SYSTEM NOISE
Noise is perhaps one of the most important limitations affecting any imaging system, and this is no different for CCD and CMOS sensors in digital cameras. There often exists a misconception that because the output is digital that it is more accurate, or not subject to noise in the same manner as in an analogue device. The measures and techniques described throughout this chapter are equally applicable to film and digital imaging sensors – the obvious difference being that working with photographic emulsions one has to digitize the response before analysis can take place.
In a similar manner to a photographic emulsion, digital imaging sensors have unique sources of noise, a number of which are described below.
Photon shot noise. A perfect detector will be subject to the limitations of the light falling on it. As mentioned previously, light randomly arrives at the sensor and at any one moment in time there will be a random deviation from the mean level. As given above, the variance of a nominally uniform patch containing N quanta will be given by 
The signal-to-noise of the idealized exposure will then be:
Thus, a perfect exposure of 7000 quanta will have an S/N or approximately 
Dark current or thermally generated noise. This is output that occurs without an input. It is caused by thermal generation of electron–hole pairs and diffusion of charge and can be reduced by cooling the device. Dark current exhibits fluctuations, similar to shot noise, so while the dark current average value can be subtracted from the output to provide the signal due to photoelectrons, the noise cannot. Termed dark current uniformity, the rate of dark current generation will also vary from pixel to pixel. A rule of thumb is that dark current approximately doubles for every 8°C increase in temperature. Therefore, cooling using liquid nitrogen or a Peltier device can reduce this significantly.
Readout noise. This occurs as the signal negotiates the readout circuitry necessary to convert it from an analogue signal to digitized values. This is sometimes combined with amplification and quantization noise. It should be noted, however, that even if amplification and quantization were perfect, some readout noise would exist. This is due to subtle variation in addressing circuits and wiring, and addition of unwanted random signals, and is not related to exposure level.
Amplification noise. This is a combination of two sources, white and flicker noise, and is dependent upon sample rate. White noise generally occurs at higher sample rates whereas flicker, sometimes know as 1/f noise, occurs at lower rates. As CMOS sensors generally have additional amplification on pixels, this can be a source or variation between pixels.
Reset noise. Before the integration of the charge for the next exposure, the photoelement is reset to a reference voltage. Each time the photodiode is reset there is a slight variation which is added into the signal.
Pixel response non-uniformity (PRNU). Each pixel will exhibit a slight variation in sensitivity to a uniform exposure. This is due to natural variation in the manufacturing process. Exposing the sensor to a uniform source will create a flat field which may be used to correct the image.
Defects. Whilst a working pixel may exhibit PRNU, defective pixels do not effectively detect a signal. Hot pixels saturate very quickly, generating maximum output regardless of the input signal. Dark or dead pixels do not respond to light. Most sensors incorporate defect detection and correction, replacing the defect value with one interpolated from the surrounding pixels. If the sensor has sufficient on-board memory, a defect map may be created to store the locations. In CCDs the defective pixels may also act as a trap and inhibit the transfer of the signal. More seriously, entire columns, rows and larger areas can be affected.
Fixed pattern noise. As the name suggests, this is noise that is consistent from frame to frame. This can include noise from a lot of effects and often defects are included in the description. Slight variations in Bayer filters, dust and manufacturing processes can all contribute to the effect (Figure 24.10).
Clock signals and other interference. Especially in the case of CCDs, reading the image so that it may be output from the sensor requires accurate timing. The signals generated from the clock may sometimes interfere with the image. Interference from other electrical devices, such as motors, and static discharge can also affect the signal. This effect will be familiar with the users of electric drills.
Figure 24.10 An enlarged example of fixed pattern noise and defective pixels from a commercial digital stills camera. Contrast has been enhanced to make the noise more apparent.
It is worth noting that for a number of reasons noise does not have to be uniform across the field of view of a particular sensor. Charge transfer efficiency was mentioned in Chapter 9. Signals from those pixels that are furthest from the analogue-to-digital converter (ADC), thus requiring more transfers, will be subject to more charge transfer loss, therefore creating one source of asymmetry. In addition, Chapter 9 details the use of infrared filters, whose cut-off frequency is dependent on the angle in incoming light. This causes fall-off of the red channel towards the edge of the field of view. Lens vignetting and the cos4 law (Chapter 6) further cause the signal to be reduced for all channels (Figure 24.11). A common method for correcting this is to use flat-field correction. By exposing to a uniform light, the relative fall-off for each channel may be observed and corrected using the inverse. This will have the effect of amplifying the signal more at the edge of the field of view of the sensor, effectively increasing the noise and reducing the dynamic range in these areas.
Assuming noise sources to be uncorrelated, the total system noise sSYS can be obtained by adding together the individual noise powers. This requires that all noise sources be specified in equivalent output units (e.g. electrons or volts). For example:
where σ2DARK = dark current, σ2PATTERN = fixed pattern noise, σ2READOUT = reset noise (from charge reading) and amplifier noise, and σ2ADC = quantization noise generated by the analogue-to-digital conversion, as detailed in Chapter 9.
Figure 24.11 An example of lens vignetting from a commercially available digital stills camera. Contrast has been increased to make the vignetting more apparent.
In addition, the noise at the output, σ2TOT will include a contribution from the input photon noise σ2q so that:
RESOLUTION, SHARPNESS AND MTF
Resolution, sharpness and MTF are all related to microimage (spatial) properties of a system, but they are not equivalent and cannot be used interchangeably, as is often mistaken. Resolution describes the finest detail that may be recorded (or reproduced) by a system and depends on the basic imaging element (the point spread function, PSF), whereas sharpness essentially describes the recording of edges, or transitions, and depends on the ability of a system to reproduce contrast, especially at high spatial frequencies (see also Chapter 19). Figure 24.12a shows an image with high resolution but low sharpness and Figure 24.12b an image with low resolution and high sharpness.
When an image is presented to a viewer, the subjective impression of sharpness and resolution will depend largely on the viewing conditions and the context in which the image is shown. Therefore, whilst a high resolution or a sharp image may suggest more accurate recording of high spatial frequencies, the notion of high spatial frequency is relative. Additionally, depending on the level present, image noise may either mask, or in some cases enhance, the subjective impression of sharpness and detail.
Resolving power of photographic systems
A good example of a signal-to-noise-based decision is illustrated by the traditional measure of photographic resolving power. Generally, resolving power measures the detail-recording ability of systems, i.e. an objective measure of resolution. In silver-based photographic systems it is determined by photographing a test object containing a sequence of geometrically identical bar arrays of varying size, and the smallest size in the photographic negative, the orientation of which is recognizable by the eye under suitable magnification, is estimated. The spatial frequency of this bar array, in line pairs per millimetre, is the resolving power. Figure 24.13 shows some typical test objects.
The test object is imaged on to the film under test either by contact printing or, more usually, by use of an optical imaging apparatus containing a lens of very high quality. A microscope is used to examine the developed negative. The estimate of resolving power obtained is influenced by each stage of the complete lens/photographic/microscopic/visual system, and involves the problem of detecting particular types of signal in image noise. However, provided the overall system is appropriately designed, the imaging properties of the photographic stage are by far the worst, and one is justified in characterizing this stage by the measure of resolving power obtained.
Figure 24.12 An image with high resolution but low sharpness (a), and low resolution and high sharpness (b).
Because of the signal-to-noise nature of resolving power, its value for a particular film is a function of the turbidity of the sensitive layer, the contrast (luminance ratio) and design of the test object, and the gamma and graininess of the final negative. It is influenced by the type of developer, degree of development and colour of exposing light, and depends greatly on the exposure level. Figure 24.14 shows the influence of test object contrast and exposure level on the resolving power of a typical medium-speed negative material.
Figure 24.13 Typical resolving-power test charts. (a) Sayce chart. (b) Cobb chart.
The chief merit of resolving power as a criterion of spatial image quality lies in its conceptual simplicity, while taking account of several more fundamental properties of the system, including the MTF of the emulsion and optics, the noise power spectrum of the developed image, and the qualities and limitations of the human visual system. Very simple apparatus is required and the necessary readings are straightforward to obtain. Although it is not a measure of image quality, resolving power has been widely used as an indicator of the capabilities of photographic materials in reproducing fine image structure. It is important, however, to note that resolving power measurements, unlike MTF measurements (see Chapter 7 and later in this chapter), cannot be cascaded through the imaging components of a chain, i.e. the resolving power of the chain cannot be calculated from the resolving powers of the individual imaging components.
Measuring modulation transfer functions
The MTF, introduced in Chapter 7, when combined with noise power measurements, provides more information concerning the response of an imaging system than does resolving power alone. There is, however, an associated increase in the difficulty of implementation and interpretation of results. The sophistication of the experimental set-ups that are needed will generally be higher and more processing steps will need to be accomplished with higher data volumes before results are available. Readers are encouraged to identify the final use of the derived information to help with the decision as to which imaging system evaluation methodology is best to use.
Figure 24.14 Variation of resolving power with exposure using test charts of increasing contrast (a–c).
To evaluate MTF there are two main measurement classifications:
• Wave recording
• Edge methods.
Wave recording – sine-wave method
This is a fairly traditional approach for the analysis of optical and photographic systems. A chart containing a range of spatial frequencies (i.e. sinusoidal intensity patterns varying in one orientation) is recorded by the system under test. Figure 24.15 illustrates such a target. The dynamic range of the signal (i.e. the exposure range) is made low enough to ensure effective linearity, i.e. linear input-to-output intensity is required. Alternatively, the input-to-output non-linearity is first corrected (the inverse relationship is applied to linearize the ouput signal) before the image modulation is determined.
For each frequency the image modulation is determined from a scan of the image. This is divided by the corresponding input modulation to obtain the modulation transfer factor (see Chapter 7). A plot of modulation transfer factor versus spatial frequency is the MTF of the system. Note that the frequency quoted is usually specified in the plane of the detector. Therefore, magnification of the test target needs to be accounted for. Multiplying the test target frequency by the magnification gives the frequency in the plane of the detector (see Eqn 24.17).
In sampled (digital) systems it is increasingly difficult to produce results up to the Nyquist limit with the sine-wave method due to phase and noise issues. At high spatial frequencies, the sensor’s elements are rarely perfectly aligned and in-phase with the target’s periodic intensity patterns. Image noise worsens this problem.
Figure 24.15 An example of a calibrated sine wave chart, M-13-60-1x, used to evaluate MTF.
Manufactured by, and courtesy of, Applied Imaging Inc., www.appliedimage.com. See bibliography for full information.
Wave recording – square-wave methods
Because of the ease of production of accurate square-wave objects, so-called square-wave frequencies are often employed. Using the same steps as in the sine-wave method, the square-wave MTF, MSQ(u), is constructed. Fourier series theory gives the conversion to obtain the correct MTF as:
where u is the spatial frequency of the square wave.
Edge input method
This method is important because it requires no special patterns for input. Theoretically, a ‘perfect edge’ contains an infinite number of frequencies and therefore it is the perfect input signal for testing the spatial frequency response of a system. Edges are commonplace objects in most scenes.
Figure 24.16 Derivation of an MTF from an edge spread function.
The main disadvantage of the method lies in its sensitivity to noise, because of the small area of image used. Again, the edge target needs to be of low contrast to ensure effective system linearity, or the recorded image needs to be corrected, as with the sine-wave method, before any calculations.
The method involves scanning the image of the edge to produce the edge spread function (ESF; see Chapter 7). This is differentiated to obtain the system’s line spread function (LSF). The modulus of the Fourier transform of the LSF, normalized to unity at u = 0, is the required MTF. The procedure is illustrated in Figure 24.16. The measured MTF contains the spatial frequency content of the edge target. It can be corrected, if necessary, using the cascading property, described later.
It should be noted that noise will influence the highest frequencies the most, because of the differentiation stage. Even if the ESF is smoothed prior to differentiation, the uncertainty in the resulting MTF still remains and will generally cause the MTF to be overestimated at higher spatial frequencies.
The edge method, whilst useful, suffers from difficulties when applied to sampled (digital) systems. Relative alignment of the edge with the sensor array will cause variations in the result due to aliasing and phase effects. Any angular misalignment results in softening of the edge when column averaging is performed. Additionally, MTFs may only be calculated to the Nyquist frequency because of the sampling theorem.
Slanted edge method
As a response to the above difficulties in MTF evaluation of sampled systems, the slanted edge technique was developed and has since been adopted as an ISO standard (ISO 12233:1999).
The technique is illustrated in Figure 24.17. An edge (A–B) is projected on to the detector at a slight angle to the vertical. The output from any one row will be an under-sampled edge trace. However, if successive rows are combined in the correct interlaced order, we obtain a single edge trace that has been very closely sampled. This trace is then differentiated and the modulus of Fourier transform is taken to yield an accurate measure of the horizontal spatial frequency response (SFR), measured well beyond the practical Nyquist limit.
The above response is named SFR, as opposed to MTF, as it takes no account of the spatial frequency content of the edge target, or of attempts to separate system components. Practically, however, the target has little effect on the result if imaged at a low magnification (e.g. 1/50th). A standard target for the measurement is available. Standard test targets for measuring SFR are shown in Figure 24.18a and b. A typical digital camera SFR is illustrated in Figure 24.19. A number of commercial pieces of software have plug-ins available to calculate SFR. The method has become popular because of its ease of use and the software tools available. SFR may be treated as MTF by correcting for the target using the cascading property below.
Figure 24.17 Schematic of CCD/CMOS detector array and illustration of the principle of the slanted edge method for SFR measurement.
Figure 24.18 (a) The test chart used for ISO 12233 SFR measurement and (b) the test target specified for reflective scanners in ISO 16067-1. The test target shown in (b) has lower contrast and therefore some advantages for systems with non-linear components.
Permission to reproduce extracts from ISO 12233 and ISO 16067-1 is granted by BSI. Test target example in (b) manufactured by, and courtesy of, Applied Imaging Inc, www.appliedimage.com. See bibliography for full information.
Figure 24.19 A typical SFR for a commercial digital camera with pixels of 9 mm in size. Also indicated is the Nyquist frequency, beyond which aliasing can occur if not controlled.
MTF/SFR provides useful measures for system design and analysis. Generally, the point at which the MTF/SFR drops to 10% will be measured as the resolution limit. Algorithm designers will also use the MTF/SFR to indicate where digital amplification of the signal may result in objectionable noise. MTF will often be used as a design criteria with respect to position in the field of view to specify lens designs.
MTF correction
With all the above methods, the measured MTF will generally be that for a system containing a chain of processes. For example, in the analysis of photographic film some form of scanning device is required to extract the spread function. When measuring the MTF of a CCD array, a lens will normally be included.
Each of these processes has its own MTF. The cascading property states that these multiply together to produce the system MTF. This is provided that the processes and the links between them are linear. Therefore, coherent optical systems will not cascade in this manner.
The effect of an individual process may therefore be removed from the overall measurement by dividing by that MTF (at least, up to the frequency at which noise dominates). Figure 24.20 illustrates this procedure for a digital acquisition and display chain.
It should be noted that the correction becomes unreliable in areas where any component MTF curve approaches zero or small values relative to the other components, as it forces the resultant curve towards infinity. A pragmatic approach is to estimate the error in the result as proportional to the reciprocal of a lowest value in any component curve at each spatial frequency.
Figure 24.20 The relationship between the MTF of a display device and the complete imaging chain, assuming linearity.
Resolving power and MTF for optical systems
It is sensible to test any optical system, usually to estimate the available maximum and minimum resolving power (RP; see Chapter 6) values and threshold contrast. A complete system can be evaluated together with any environmental effects such as vibration, different systems can be compared and a check kept on the maintenance of an agreed or necessary performance. Methods of testing can either be by field testing or be studio based.
Field testing is concerned with: actually taking pictures under typical user conditions to evaluate human and environmental factors; the type and quality/quantity of subject lighting; latent image effects for delayed processing; the effect of subject matter on image quality, and of subject distance or movement. Subjective judgements of the final image can be scaled on the basis of either a three-point Pass/Acceptable/Fail system or a quality scale from 1 to 5.
Field and studio testing can use a collimator, a compact optical instrument consisting of a well-corrected lens with a pinhole or illuminated test reticule at its rear focal plane. Viewing from the front gives the reticule at an artificial infinity. An autocollimator has a beamsplitter for direct viewing of a reticule in the focal plane, or on the film, photocathode or CCD array. It is possible to check focal length and focusing scale accuracy as well as misalignments and aberrations. Field use allows quality control of a complete system of lens, mount and camera body.
Studio-based resolution testing is easy and cheap without specialist equipment. Limitations include use of finite conjugates, repeatability no better than 10% and that no single figure of merit is given. The two basic methods are either to photograph with the lens under test a reduced image (m = 0.02) of a planar array of test patterns (targets) covering the field of view, or use the lens to project an enlarged image (m = 50–500) on to a screen of a precision micro-image array of targets on a plate placed in the focal plane of the lens. The former method gives a result for the composite performance of the system, including detector, focus, vibration and image motion.
Various designs of test target are used, including the Sayce chart, Cobb chart and Siemens Star. The USAF (1951) target is widely used, having a three ‘bar’ design with bar and space of widths d and length 5d. Target elements have size decreasing by factor of 
equivalent to a 12% spatial frequency (u) change, where u = 1/2d and a line pair = 2d. The RP limit is the value of 1/2d for an image in both orthogonal azimuths that is just resolved. Target frequency uo and image frequency ui are related by magnification m:
Target contrasts used are 1.6:1, i.e. ∆D = 0.2; 6.3:1, ∆D = 0.8; and 100:1, ∆D = 2.0, where ∆D is the reflection density.
Optical resolving power is also given in line pairs per mm (lp mm−1, lpm or mm−1) as RP = 1/2d. The results are influenced by exposure, focus, processing and visual evaluation. Imaging performance can be assessed from inspection of RP figures for a range of conditions of focus, aperture, etc.
MTF data for optical systems is very useful. A direct use is the determination of best focus setting by the maximum response obtained. Optical design software often uses models of MTF to optimize lens designs in the later stages. It is useful to remind the reader at this point that only incoherent component MTFs may be cascaded; the phase of the light must be random. Practically, this means that for a multi-element lens, the MTF of the lens as a whole is not the MTF of the individual elements multiplied together.
A single MTF curve does fully describe lens performance and many curves are needed for different apertures, in monochromatic and white light, and at different focus settings, conjugates and orientations. Lenses can be compared directly under similar conditions and characterized from their MTF data as being, say, ‘high image contrast but limited RP’ or ‘moderate contrast but high RP’. Both types give distinctive images and are preferred for specific types of practical work, typically general-purpose recording by the former and copying by the latter. Separate assessments have to be made of other image characteristics such as distortion and flare.
Sharpness, MTF and image quality
It is seen throughout this book – particularly in Chapter 19 – that image quality involves psychophysical processes acting on various physical attributes of the image. The MTF summarizes those attributes that contribute to such subjective impressions as sharpness and resolution.
The term ‘sharpness’ refers to the subjective impression produced by an edge image on the visual perception of an observer, produced by either film or a digital sensor. The degree of sharpness depends on the shape and extent of the edge profile, but being subjective the relationships are complex and difficult to assess. Traditionally, an objective measure of photographic sharpness, known as acutance, can be calculated from the mean-square density gradient across the edge. This involves very careful microdensitometry for photographic-type processes and is subject to high error. The result only partly correlates with visual sharpness.
The density profile of a photographic edge depends on the turbidity of the emulsion and on the development process. When adjacency effects are significant, enhanced sharpness often results. This phenomenon is used to advantage in many developer formulations, and is illustrated in Figure 24.21a. The density in the immediate neighbourhood of the high-density side of the edge is increased by the diffusion of active developer from the low-density side. Development-inhibiting oxidation products diffusing from the high-density side reduce the density in the immediate neighbourhood of the low-density side of the edge. The luminance overshoots created at the edge boundaries in the final image produce the sensation of increased sharpness.
The same approach and phenomenon form the basis of analysing digital imaging sensors. An edge profile may be imaged, after which digital values may be inspected to form ‘traces’, as above. In a similar manner, image-sharpening procedures used in digital image processing (see Chapter 27), such as the crispening operator, based on the Laplacian, create the sensation of increased sharpness by producing similar overshoots in the grey levels of pixels close to edges.
It was noted earlier that the line spread function of a system can be obtained from the first derivative of the edge profile. This suggests that there could be a relationship between sharpness and MTF.
Figure 24.21 Edge traces for a typical medium-speed film in the presence (a) and absence (b) of edge effects.
Figure 24.22 The effect of blurring chrominance and luminance channels. (a) The original image. (b) Blurring only the chrominance information. (c) Blurring only the luminance information. The chrominance information is blurred using a Gaussian kernel 10 times wider than that of the luminance, yet the luminance degradation is more apparent.
In recent years much research has been conducted to establish models of image quality. In many cases MTF curves feature prominently in the derived expressions. The MTFs are for components of the imaging chain, including the human eye and visual system, and are weighted according to factors such as the type of image and the viewing conditions. As a simple example of this weighting process, consider an imaging system in which the luminance channel is separate from the colour channels. It is well known that the luminance MTF is more important than the colour MTFs as far as the perceived quality of the resulting images is concerned. In other words, provided that the greyscale image component is sharp, the final image can appear acceptable even if the colour component is blurred (Figure 24.22). Much recent research effort has been applied to the development of spatial-chrominance-based models of the human visual system, including the effects of visual masking, leading to better performing image quality metrics (see Chapter 19). This is only possible through the advancements in computing and numerical processing.
The value of quality metrics involving the use of MTF is that they can provide a very useful engineering link between the subjective human response and the need for hard engineering goals in the design of imaging products. It is becoming increasingly common for metrics to be used to estimate a required MTF to achieve a desired level of perceived image quality. The MTF may then be used as a design constraint in lens design software or for ‘pixel engineering’. Further, designing the MTF of lenses to match the CCD or CMOS sensor for which they are intended reduces aliasing and, in some cases, manufacturing cost. Because of its flexibility, MTF theory is considered to be one of the most important tools available to the image scientist.
DETECTIVE QUANTUM EFFICIENCY (DQE)
General considerations
Imaging systems work by detectors responding to light energy. The information contained in an exposure can in principle be measured in terms of the number and distribution of energy quanta in the exposure.
The efficiency of an imaging process in recording the individual quanta in an effective manner is therefore a valuable measure, particularly in applications such as astronomy, where exposure photons may be in short supply. Such a measure is detective quantum efficiency (DQE). Strictly it is a function of spatial frequency, involving the MTF and normalized noise power, together with a quantity 3(0), which represents the effective fraction of quanta counted. These three parameters are responsible for degrading the image and thus lowering the information recording efficiency.
If we assume our exposures are low frequency, the MTF and normalized noise power are both unity, DQE becomes identical to the quantity 3(0). This is defined in terms of signal-to-noise ratios as follows:
where (S/N)2OUT is the squared signal-to-noise ratio of the image and (S/N)2IN is the squared signal-to-noise ratio of the exposure. (S/N)2OUT takes into account all the system inefficiencies associated with the effective recording of quanta (random partitioning of exposure, quantum efficiency of detector, random processes in amplification, etc.). It must be expressed in equivalent input exposure units. (S/N)2IN depends on the number of quanta in the exposure.
Figure 24.23 (a) Representation of a photon exposure pattern following an exposure time of T seconds. (b) Image where all photons are recorded but an MTF operates. (c) Typical image from real system where an MTF operates and only a fraction of the exposure photons are recorded.
An ideal imaging system would have DQE = 1 (100%). Real imaging systems have DQE < 1, e.g.:
| TV camera (Orthocon) | DQE ~ 0.1 |
| CCD cooled | DQE ~ 0.5 |
| CCD uncooled | DQE ~ 0.3 |
| CCD Back thinned | DQE ~ 0.8 |
| Photographic emulsion | DQE ~ 0.01 |
| Human eye | DQE ~ 0.03. |
The concept of DQE can be illustrated with the help of Figure 24.23. In Figure 24.23a, a representation of an input photon pattern is shown. It comprises a disc of area A μm2 and intensity above background of IS photons μm−2 σ°1. The background intensity is IB photons μm−2 σ°1 on average. Suppose an exposure lasts for T seconds. The total number of photons contained in the disc (the signal strength) will be AIST. The mean number of photons present in an equivalent area A of the background is AIBT. Because the distribution of photons follows Poisson statistics, the background is accompanied by a noise level of 
. The signal-to-noise ratio for this exposure is given by:
In other words, the signal-to-noise ratio increases as the square root of the exposure time. Since the background exposure level is just q = IBT photons μm−2 we can substitute for T in Eqn 24.32 to obtain:
where k is a constant for the particular photon pattern, i.e. the squared input signal-to-noise ratio is proportional to the exposure level.
An ‘ideal’ imaging system would record the positions of each photon contained in this input pattern, and display an image identical in all respects. In other words, the image or output signal-to-noise, (S/N)OUT, would equal (S/N)IN.
Figure 24.23b shows an image of Figure 24.23a in which all quanta have been uniquely recorded, but the image elements corresponding to their detected positions have been moved, at random, a small distance to simulate light diffusion. This corresponds to an MTF that is not unity over the range of spatial frequencies contained. The disc is still visible although its edges are less well defined. In this example the signal-to-noise ratio is only slightly less than that for the exposure.
Figure 24.23c illustrates an image typical of a real process where only a fraction of the quanta is effectively recorded after light diffusion. The system amplifies the image elements to render the image visible. The image is now very noisy and the star is barely detectable. The signal-to-noise ratio for this image is very much less than that for the exposure. In terms of signal-to-noise, this image could be matched by an image from an ‘ideal’ system operating at a much lower background exposure level (q′ say). In other words, although the actual exposure level is q, the result is only ‘worth’ the lower exposure level q′.
If the signal-to-noise ratio for this image is denoted (S/N)OUT, REAL, we can write:
Using Eqn 24.21, we can write the DQE for the real system as:
Equation 24.22 indicates that the DQE of a real image recording process at an exposure level q is given by the ratio q′/q, where q′ is the (lower) exposure level at which the hypothetical ‘ideal’ process would need to operate in order to produce an output of the same quality as that produced by the real process.
DQE for the photographic process
By expressing the signal-to-noise ratio of the photographic image in terms of sensitometric parameters, we can derive the following working expression for the DQE of a photographic emulsion:
Here γ2 represents the slope of the photographic characteristic curve at the exposure level q quanta μm−2, log10e is a constant and Aσ2 is the image granularity at the exposure level q.
Equation 24.32 is equivalent to writing:
Such a relationship, happily, agrees with conclusions arrived at from a more intuitive commonsense approach to the problem of improving the quality of photographic images.
Figure 24.24 shows a DQE vs. log exposure curve for a typical photographic material. It can be seen that the function peaks around the toe of the characteristic curve and has a maximum value of only about 1% at best. Either side of the optimum exposure level, the DQE drops quite rapidly. This very low DQE is a characteristic of all photographic materials. They are not particularly efficient in utilizing exposure quanta. Fortunately, this is not significant in normal photography where exposure levels are high enough to ensure an adequately high output signal-to-noise ratio. In situations involving low light levels, however, such as may be encountered in astronomical photography, this low DQE is a severe limitation, and has led to the development of far more efficient image-recording systems to replace or supplement the photographic material.
Figure 24.24 Detective quantum efficiency (DQE) as a function of exposure for a typical photographic material. The function peaks at an exposure level near the toe of the characteristic material.
DQE for a CCD or CMOS imaging array
CCD and CMOS imaging arrays are now the detector of choice in most imaging applications. So-called megapixel cameras are available that offer MTF characteristics comparable to those of photographic emulsions. They also exhibit significantly higher DQE than the photographic emulsion. In this section we introduce the processes that determine the DQE for such a device.
We begin by defining some necessary terms with reference to Figure 24.25, which shows the transfer function of a CCD array prior to analogue-to-digital conversion.
Responsivity, R. This is the slope of the straight-line section of the output–input transfer curve. Possible units are volts/joules cm−2, volts/quanta mm−2, etc. R depends on a number of factors (wavelength, temperature, pixel design) but can be broadly expressed as:
where g is the output gain (volts per electron) and h is the quantum efficiency.
Saturation equivalent exposure (SEE). This is the input exposure that saturates the charge well. If the dark current is considered negligible then we have (approximately):
where Vmax is the full-well capacity in volts and R is responsivity.
Noise equivalent exposure (NEE). This is the exposure level that produces a signal-to-noise ratio of 1. If the measured output noise is sTOT (rms) and if the dark current is considered negligible, then:
Sometimes the NEE is used as a measure of the sensitivity, S, of the detector.
Figure 24.25 Characteristic curve of a CCD/CMOS array prior to analogue-to-digital conversion.
DQE and signal-to-noise ratio
Consider an exposure level q photons per unit area. The noise associated with this is q1/2 photons per unit area (because photon arrival in space and time is governed by Poisson statistics).
Therefore, the signal-to-noise ratio of the exposure, (S/N)IN, is:
so
This exposure will yield ηq photoelectrons per unit area and gηq volts. The noise at the output due to these photoelectrons will be σq = (ηq)1/2 σ°rms or g(ηq)1/2 volts rms. The variance is then σ2q = g2ηq (using volts as the output unit).
The total output noise is therefore:
We must now express this output noise in terms of the input exposure units (i.e. we transfer the noise level back through the transfer curve):
Hence:
and
Note that if there is no system noise, then the low-frequency DQE ε(0) = η.
Example
A photodetector has a quantum efficiency η = 25% and system noise σSYS equivalent to 20 electrons rms per unit area. What is the low-frequency DQE ε(0) at an exposure level of (a) 1000 photons per unit area and (b) 105 photons per unit area?
Solution
Since output noise is given in terms of electrons per unit area, we can set g = 1 in Eqn 24.32:
The DQE function of Eqn 24.32, with the parameters of this example, is plotted in Figure 24.26. The curve is significantly higher than the peak DQE for typical photographic materials. For high exposure levels (up to the SEE), the DQE approaches the quantum efficiency of the device.
Figure 24.26 The DQE of a CCD/CMOS imaging array. For high exposures (limited by the SEE) the DQE approaches the quantum efficiency, which in this example is 25%.
INFORMATION THEORY
Information theory supplies a mathematical framework within which the generation or transmission of information can be quantified in an intuitive manner. In other words, information can be measured precisely and unambiguously in a way easily understood and applied in practice. The reader is also directed to Chapter 29, where some of the background to the concepts described here (in particular the work of Claude Elwood Shannon, from which much of the theory originated) is further discussed.
The process of gaining information is equivalent to the removal of uncertainty. We speak of a source of information containing a number of possible outcomes. The uncertainty in the source depends on the probabilities of the individual outcomes. When a particular outcome is generated, that uncertainty has been removed. We have gained information. The uncertainty in a source is measured using the concept of source entropy. If a source has n possible outcomes, and the ith outcome occurs with probability pi, then the source entropy is defined as:
where the unit, bit, takes the same definition as in Chapter 9. If the outcomes are all equally likely, the source entropy reduces to:
In a sequence of independent trials, where each trial generates an outcome, we gain H bits of information per outcome on average.
Information capacity of images
Suppose we have an image containing m independent pixels per unit area where each pixel can take on one of n different values (assumed equally likely). Each pixel can be considered as a source of information. According to Eqn 24.32 the entropy per pixel is given as H = log2 (n) bits. Regarding the image as an information channel, this means that each pixel is capable of carrying log2 (n) bits. We say that the information capacity, C, is log2 (n) bits per pixel or, more generally:
As an example, consider a CCD image array comprising 1800 pixels horizontally by 1200 pixels vertically. If each pixel can independently take on one of 256 different levels, the information capacity of one frame is given as:
The information capacity of a photographic emulsion is less easy to evaluate, since there is no readily identified pixel. The area of the point spread function at 10% has been used as an equivalent area. The number of recording levels, n, depends on the level of granularity. Such calculations yield values for the information capacity of camera speed materials of the order of 105 bits cm−2. A 35 mm frame will therefore have a capacity of approximately 8.5 ×106 bits. It would appear that the digital system has overtaken the photographic process with respect to this measure of performance.
In practice the above information capacities are rarely achieved. To realize the idealized capacity in such a system, each pixel and intensity level in the system would be independent and noise free.
Correlation of neighbouring pixels occurs in images due to the subject being imaged. Therefore, the information content for a typical image will be much less than the information capacity, since for most meaningful images many neighbouring pixel values will be the same. We say there is redundancy.
Further correlation, not caused by the original signal, is also present and for digital devices may be attributed to the performance of the optics: Bayer-type colour array, interpolation, denoising algorithms, anti-aliasing filters and system electronics. This component may essentially be represented by the PSF of the imaging system as a whole and its width will indicate the extent to which correlation between neighbouring pixels exists. As the PSF widens, the number of independent imaging units within the image drops. Conversely, it may be imagined that if the combined PSF of all other components was smaller than a single pixel, the output would be ideal. Correlation amongst pixels in some digital devices can extend tens of pixels and thus the size of an imaging cell would be of that order.
Therefore, regardless of the number of pixels actually output by a given device, the PSF will more accurately represent the true number of independent imaging units or cells. This effect is important for purchasers of digital imaging equipment to consider. The reported number of pixels of the device will not necessarily relate to the final resolving power of the image. Certainly, if the pixel size is much smaller than the Airy disc of the accompanying lens, the value of the ‘extra’ pixels should be evaluated. A good 6 Mp digital camera can outperform a poor 10 Mp device.
The number of independent levels that an imaging unit may take is dictated by the noise present in the system. As noise in the image rises, it is increasingly difficult to predict with accuracy the original intensity of the subject. Thus, the number of independently measurable levels in the output falls. It should also be noted that a component of pixel correlation may be added by the effects of noise and may be assessed by evaluating the noise power spectrum of the device. Certainly, a more sophisticated treatment of information capacity as provided by Shannon defines the signal-to-noise as the ratio of signal-to-noise spectral powers at each spatial frequency, partially accounting for correlated noise effects.
Information capacity may be assessed for any system for which a reasonable estimate of the PSF and noise can be made. An important example of its application is in the context of image compression. Both information theory and compression are covered in more depth in Chapter 29; therefore, the following is mentioned briefly here as an illustration. Figure 24.27 shows the information capacity of JPEG 6b compressed images versus compressed file size. By treating blocking artefacts in JPEG compression as noise and evaluating the average PSF of the entire image using a comparison of pre-and post-compressed information, it is possible to calculate the information capacity at different quality factors. It may be seen that as information is discarded because of the compression of the image, the information content of the decompressed image also decreases.
Figure 24.27 JPEG information capacity versus compressed file size.
Increasing interest in the information capacity of imaging systems exists, as it has been proposed that there is a degree of correlation between it and the perceived quality of images. In today’s technology and socially networked society, we are looking to compression and processing techniques to intelligently handle and transmit images in systems such as cable TV, DVDs and the Internet. The use of information capacity allows these compression and processing techniques to be optimized, yielding the best possible image quality.
BIBLIOGRAPHY
Bolton, W., 1995. Fourier Series. Longman, London, UK.
Bracewell, R.N., 1999. The Fourier Transform and Its Applications, third ed. McGraw-Hill, New York, USA. Castleman, K.R., 1996. Digital Image Processing. Prentice-Hall, Englewood Cliffs, NJ, USA.
Dainty, J.C., Shaw, R., 1974. Image Science. Academic Press, New York, USA.
Gleason, A. (translator), et al., 1995. Who Is Fourier? A Mathematical Adventure. Transnational College of LEX/Blackwell Science, Oxford, UK.
Gonzalez, R.C., Woods, R.E., 2008. Digital Image Processing, third ed. Addison-Wesley, Reading, MA, USA.
Gonzalez, R.C., Woods, R.E., Eddins, L.C., 2003. Digital Image Processing Using MATLAB®. Prentice-Hall, Englewood Cliffs, NJ, USA.
Goodman, J.W., 1996. Introduction to Fourier Optics (Electrical and Computer Engineering), second ed. McGraw-Hill, New York, USA.
Hecht, E., 1987. Optics, second ed. Addison-Wesley, Reading, MA, USA.
Holst, G.C., 1998. CCD Arrays, Cameras and Displays, second ed. SPIE Optical Engineering Press, Bellingham, WA, USA.
ISO/FDIS12233:1999(E) 1999. Photography – Electronic Still Picture Cameras – Resolution Measurements. International Organization for Standardization, New York, USA.
James, J., 1995. A Student’s Guide to Fourier Transforms with Applications in Physics and Engineering. Cambridge University Press, Cambridge, UK.
Proudfoot, C.N., 1997. Handbook of Photographic Science and Engineering, second ed. IS&T, Springfield, VA, USA.
Smith, E.G., Thomson, J.H., 1988. Optics, second ed. Wiley, Chichester, UK.
Permission to reproduce the M13-60-1x Sine Pattern test target in Figure 24.15, and the example ISO 16067-1 target in Figure 24.18b was granted by Applied Image Inc. Readers should note that an ISO 12232 target, as shown in Figure 24.18a with an extended spatial frequency range is also available. These and other targets are manufactured by Applied Image, Inc., 1653 East Main Street, Rochester, NY 14609, USA. Phone: +1 585 482 0300, Fax: +1 585 288 5989, E-mail: [email protected] website: www.appliedimage.com.
Permission to reproduce extracts from ISO 12233:2000 Photography–Electronic still-picture cameras–Resolution measurements, and ISO 16067-1:2003 Photographye Spatial resolution measurements of electronic scanners for photographic images–Part 1: Scanners for reflective media, for Figure 28.18, is granted by BSI. British Standards can be obtained in PDF or hard copy formats from the BSI online shop: www.bsigroup.com/Shop or by contacting BSI Customer Services for hard copies only: Phone: +44 (0) 20 8996 9001, E-mail: [email protected].