JAI Image Classes

The javax.media.jai.PlanarImage class is an abstract class that also implements the RenderedImage interface, so any concrete PlanarImage subclass has the capability to provide image data in a Raster to objects requesting its data. The PlanarImage class doesn't implement the WritableRenderedImage interface, but one subclass that does have this functionality is the javax.media.jai.TiledImage class. The TiledImage class is the main class for performing image processing directly on pixel data. Another important PlanarImage subclass is the javax.media.jai.RenderedOp class. This class doesn't implement the WritableRenderedImage interface, but does provide methods for data creation through a set of operators.

Thus, a TiledImage object allows you to edit the pixel data yourself, whereas a RenderedOp object edits the pixel data for you according to its corresponding operator. We will examine these three classes in detail starting with the PlanarImage. At the end of this chapter, we will examine two other PlanarImage subclasses: RenderableOp and ImageOp. The RenderableOp class performs a function similar to the RenderedOp class except that it is meant to be used with rendering independent RenderableImages instead of rendering dependent RenderedImages. This distinction will be made clearer when RenderableOps are discussed. The final subclass, ImageOp, is used to carry out the operations specified in the RenderedOp and RenderableOp objects. This class will be discussed in the “Extending JAI” section.

PlanarImage

In order to better understand the PlanarImage class, we will discuss the following topics:

• Image layout

• Properties

• Sources

• Tiles

Image Layout

A PlanarImage contains an object of class javax.media.jai.ImageLayout, which is used to hold information describing the image dimensions (minimumXValue, minimumYValue, width, height), information describing pixel access (SampleModel), information describing pixel interpretation (ColorModel), and information describing the tile grid layout (tileGridXOffset, tileGridYOffset, tileWidth, tileHeight). The PlanarImage class contains accessor methods so that these values can be read without going through the contained ImageLayout object.

Properties

PlanarImages not only contain pixel data, but they also contain a set of properties. These properties are often referred to as the image's metadata (see Chapter 5, “Image I/O Package,” for more information on metadata). Typical properties for a new PlanarImage are the following: image_min_x_coord, image_min_y_coord, image_width, and image_height. However, depending on the image's initial format, the number and type of initial properties can vary. As will be discussed when describing the set of statistical operators, some operations can add to this property list. For example, the "mean" operation provides a property called mean that provides the average value of each of the image's bands. The value of any property can be found as follows:

public Object getProperty(String name)

where the return value must be cast into the correct class, that is,

double[] meanValuesForEachBand = (double [])planarImage.getProperty("mean");

You can also add properties to a PlanarImage, which is a convenient way to keep created metadata with its corresponding image. This is done using the following PlanarImage method:

void setProperty(java.lang.String name, java.lang.Object value)

Sources and Sinks

With any image, there can be one or more images that were used to derive it. For example, if imageA and imageB were added to create imageC, imageA and imageB could be considered imageC's source images. In JAI, a PlanarImage object keeps a reference to its sources and because its sources are also PlanarImages, they keep references to their sources. Thus, a PlanarImage represents much more than a single image: It is part of a graph describing the history of its creation. For this reason, a PlanarImage is often considered part of a directed acyclic graph (DAG) that is a graph in which the connection between nodes is uni-directional and once you travel from one node to another, there is no way to get back again. In these JAI DAGs, each PlanarImage is considered a node and the reference from one PlanarImage to another is considered a graph edge (see Figure 6.4).

A PlanarImage's sources can be found using one of the following methods:

public java.util.Vector getSources()
public PlanarImage getSourceImage(int index)

The latter one is often used in conjunction with the following PlanarImage method:

								public int getNumSources()

A PlanarImage also has sinks, which refer to the PlanarImages that the PlanarImage helped create. The use of sinks isn't completely analogous to the use of sources because of the way JAI defines reachable nodes. This definition is important because a node that isn't considered reachable is available to be garbage collected. The definition states that any node in a DAG that has an external reference is reachable and any node that is a source to a node with an external reference is reachable. Any other node in the DAG is unreachable and can be garbage collected.

For example, referring back to Figure 6.4, if there were a reference to PImageF, all nodes in that DAG would be reachable and none would be garbage collected. On the other hand, if there were no external references to PImageF and there was an external reference to PImageE instead, PImageF would be unreachable and would eventually be garbage collected. What makes this confusing is that if PImageF hasn't been garbage collected, using PImageE's getSinks method can provide an external reference to PImageF; at which point it becomes reachable. Therefore, the difficulty of working with sinks is that depending on the DAG's external references and the efficiency of the garbage collector, a PlanarImage's sinks might or might not exist.

Tiles

Tiles are rectangular segments of a Raster that allow you to process or display particular regions of an image instead of trying to work with the entire image at once. This is very useful for large images that might not fit completely into memory. All image tiles have the same width and height, so they divide the Raster into a rectangular grid. By default, a tile has the same dimensions as its corresponding image, meaning that its Raster is composed of a single tile.

A tile contains the same pixel bands as its associated Raster and is used to access that Raster's data. One way to access this data is through the public Raster getTile(int x, int y) method, which will return the tile associated with tile index x, y. Because the returned object is a Raster and not a WritableRaster, this method can be used to get tiles for read-only purposes. The ability to provide changeable Raster data isn't part of the PlanarImage class, but is available in its TiledImage subclass. As will be described in the following section, the TiledImage class provides management of the writable tiles so that if more then one object has a reference to the same writable tile, the TiledImage object can be used to inform them all if a change is made to that data.

TiledImage

The javax.media.jai.TiledImage class is JAI's closest analogy to the BufferedImage class. You can roughly think of it as taking a BufferedImage and adding the necessary functionality to allow it to be used in the JAI package. The three constructors for TiledImages are as follows:

TiledImage(int minX, int minY, int width, int height,
         int tileGridXOffset, int tileGridYOffset,
         java.awt.image.SampleModel tileSampleModel,
         java.awt.image.ColorModel colorModel)

TiledImage(java.awt.image.RenderedImage source,
           boolean areBuffersShared)

TiledImage(java.awt.image.RenderedImage source,
           int tileWidth, int tileHeight)

where the first constructor creates a TiledImage from the provided parts, whereas the second and third constructors create TiledImages using a RenderedImage to supply these parts. The second constructor provides the added functionality of being able to share the data in the RenderedImage, whereas the third constructor allows you to retile this data as it is being copied. For examples of how these constructors are used, the first constructor is demonstrated in Listing 6.14 and the second and third constructors are demonstrated in Listing 6.3.

The TiledImage class implements both the RenderedImage and WritableRenderedImage interfaces so that it can provide read-only and writable tiles. To obtain read-only tiles, use the same method that was discussed for PlanarImage, namely getRaster. In order to edit tiles, one of the following three ways should be used.

The first way is through the public WritableRaster getWritableRaster(int x, int y) method, which returns a WritableRaster so that you can edit individual pixels or samples using methods previously discussed in Chapter 4, namely setPixel, setPixels, setDataElements, setSample, and setSamples methods. All these methods are declared in the WritableRenderedImage interface.

When using the getWritableTile method, care must be taken whenever the data contained in one of the tiles is changed. This is because the getWritableTile method doesn't make a copy of the Raster's data, so all objects that call getWritableTile using the same tile index will obtain a reference to the same writable tile. The best way to ensure that each of these objects is aware of any changes is for each object interested in a particular writable tile to register itself with the TiledImage object. Then through the use of appropriate events, each of these objects can be kept informed about any changes to that tile. The next section will describe this process in more detail.

A second way to edit tile data is to use one of the following TiledImage methods to overwrite all or part of the tile data using a Raster:

							public void setData(Raster r)
public void setData(Raster r, Roi roi)

In the first method, all regions of the TiledImage data that overlap the provided Raster will be set to the Raster's data values. All regions of the TiledImage outside the Raster's bounds will be unchanged. In the second setData method, a javax.media.jai.ROI (region of interest) object is provided. A ROI object is a single band image which contains a threshold value. All ROI pixels greater than or equal to this threshold value are considered on and all ROI pixels less than this threshold value are considered off. The way the ROI object is used in this setData method is that it is overlaid on top of the provided Raster. Then only the Raster pixels that correspond to on ROI pixels will be used to set the tile data. Thus, if the on ROI pixels make up a circle, only the tiled data corresponding to that circle will be set. All tiled data outside of that circle will be unchanged.

The last way to edit Raster data is to simply use the TiledImage's public Graphics2D createGraphics() method to obtain a Graphics2D object that can be used for drawing directly on the WritableRaster.

TiledImage Events

Because any number of objects can have an interest in a particular tile, there needs to be some mechanism for finding out if this tile's data has changed. This can be done by having these objects register themselves as java.awt.image.TileObservers using the TiledImage's addTileObserver method. In order to become a TileObserver, you must implement the java.awt.image.TileObserver interface and define the

public void tileUpdate(WritableRenderedImage source,
                       int tileX, int tileY, boolean willBeWritable)

method. Then whenever a tile is about to be updated or released, this information is sent to the TileObserver using the tile index (tileX, tileY) and a willBeWritable variable, which specifies whether that tile is about to be updated (willBeWritable == true) or if it is about to be released (willBeWritable == false). Each TiledImage object uses its getWritableTile method and its releaseWritableTile method to decide when to send out tile update events.

Basically, each getWritableTile call adds an external reference to a tile, and each releaseWritableTile call removes an external reference from a tile. Thus, a tile is considered “about to be updated” when it goes from a state in which no object has an external reference to it as a writable tile to a state in which an object has called getWritableTile for that tile.

Similarly, a tile is considered “about to be released” when it goes from a state in which at least one object has an external reference to it as a writable tile to a state in which the last object that has such a reference releases it by calling releaseWritableTile. Note that the TiledImage's setData method initially calls the getWritableTile method for each affected tile before it changes the pixel data and then calls the releaseWritableTile method for each tiles when it is done. Thus, if there are no other external references to the tile of interest, the setData method will generate two tile update events. This can be demonstrated in Listing 6.3, which uses both the getWritableTile method and the setData method to change the TileImage data. One last point is that using a Graphics2D object to write on a TiledImage will also generate tile update events for all affected tiles.

Listing 6.3 TileTester.java

package ch6;

import java.awt.*;
import java.awt.image.*;
import javax.swing.*;
import javax.media.jai.*;
import javax.media.jai.widget.*;

public class TileTester extends JFrame implements TileObserver {

    /**
       TileTester.java - takes two images of the same size and uses
       tiles from the first image to edit tiles in the second image.
    */
    public TileTester(String filename1, String filename2) {
        RenderedOp inputRO1 = JAI.create("fileload", filename1);
        RenderedOp inputRO2 = JAI.create("fileload", filename2);
        if ( (inputRO1.getWidth() != inputRO2.getWidth()) ||
             (inputRO1.getHeight() != inputRO2.getHeight()) ) {
            System.err.print("Images must have same dimensions ");
            System.err.println("for this example to run properly");
            System.exit(1);
        }

        /*
           Create two TiledImages, one for each RenderedOp.

           We are specifying the tile size as half the
           width and height of the source RenderedOps

           Thus, each TiledImage will have 4 tiles,
           tile(0,0), tile(0,1), tile(1,0) and tile(1,1);
        */
        TiledImage ti1 = new TiledImage(inputRO1,
                                        inputRO1.getWidth()/2,
                                        inputRO1.getHeight()/2);
        TiledImage ti2 = new TiledImage(inputRO2,
                                        inputRO2.getWidth()/2,
                                        inputRO2.getHeight()/2);

        //addTileObserver for the 2nd TiledImage
        ti2.addTileObserver(this);

        //ti2copy will copy data from ti2's DataBuffer
        TiledImage ti2copy = new TiledImage(ti2, false);

        //ti2share will share ti2's DataBuffer
        TiledImage ti2share = new TiledImage(ti2, true);

        /*
           Force rendering of ti2copy and ti2share.
           Rendering either will cause ti2 to be rendered.
           Displaying them will also cause them to be rendered,
           but it happens in a separate thread.  This way we
           have more control.
        */
        Raster[] tmpR;
        tmpR = ti2copy.getTiles(); // render ti2copy
        tmpR = ti2share.getTiles(); // render ti2share

        // now display the TiledImage
        getContentPane().setLayout(new GridLayout(2,2));
        getContentPane().add(new ch6Display(ti1));
        getContentPane().add(new ch6Display(ti2));
        getContentPane().add(new ch6Display(ti2copy));
        getContentPane().add(new ch6Display(ti2share));

        pack();
        show();

        /*
          take tile(0,0) from TiledImage ti1 and use it to replace
          tile(0,0) in TiledImage ti2.

          This will only effect ti2 and ti2share, not ti2copy.

          Also, the setData method will cause ti2 to generate two tile
          update events.  One for when tile with index 0,0 is about to
          become writable and one when it is about to be released.

          Both of which happen implicitly since there are no calls to
          getWritableTile or releaseWritableTile
        */

        Raster r00 = ti1.getTile(0,0);
        ti2.setData(r00);
        repaint();

        /*
          copy tile(1,1) from TiledImage t1 and use it to
          replace tile(1,1) in ti2.
          Again ti2 generates two tile update events.
          Both of these happen explicitly;
          one when getWritableTile is called and one
          when releaseWritableTile is called
        */
        Raster ri11 = ti1.getTile(1,1);
        WritableRaster wr = ti2.getWritableTile(1,1);
        wr.setRect(0,0,ri11);
        ti2.releaseWritableTile(1,1);
        repaint();
    }

    /*
      this method gets called to handle any tile update events
    */
    public void tileUpdate(WritableRenderedImage source,
                           int tileX,
                           int tileY,
                           boolean willBeWritable) {
        System.out.println("Tile("+tileX+","+tileY+")");
        if (willBeWritable)
            System.out.println(" is writable");
        else
            System.out.println(" is not writable");
    }

    /**
       input should be two filenames representing equal sized images
    */
    static public void main(String[] args) {
        if (args.length != 2)
            System.err.println("Usage: TileTester filename1 filename2 ");
        else
            new TileTester(args[0], args[1]);
    }
}

The output of Listing 6.3 will be as follows:

Tile(0,0) is writable
Tile(0,0) is not writable
Tile(1,1) is writable
Tile(1,1) is not writable
							

RenderedOp

Another important PlanarImage subclass is the javax.media.jai.RenderedOp class. The objects of this class store information necessary to carry out image processing operations. This information consists of an operation name, a java.awt.image.renderable.ParameterBlock (containing sources and parameters), and a java.awt.RenderingHints object, which provide hints for how the RenderedOp object should perform its image rendering. Each one of these items will be described in detail later in this section.

Because a PlanarImage contains a reference to each of its source PlanarImage's, a RenderedOp object contains a reference to each of its source PlanarImages (which could be RenderedOp object or another PlanarImage subclass), thus RenderedOp objects can be viewed as a DAG just as PlanarImage objects were (refer to Figure 6.4 and Figure 6.5). What is interesting about a DAG consisting of RenderedOp objects is that a particular RenderedOp object can describe the complete set of image processing operations, source images, and parameters necessary to derive its RenderedImage from the original source images.

When it is time for the final RenderedOp in the RenderedOp DAG to be rendered, it pulls the data from its sources, which in turn, pulls the data from their sources, and so on. Thus, in order for one RenderedOp node to be rendered, all the preceding RenderedOps must be rendered. Of course, if a RenderedOp has already been rendered, it will not be rerendered unless a source or a parameter is changed. (See the “RenderingChangeEvents” section for more information on how changing parameters and sources can cause a RenderedImage to be rerendered.)

When a RenderedOp is rendered, it creates a RenderedImage. This rendering usually occurs in one of two ways: with an explicit call to its getRendering method or by an implicit call to this method. This latter situation occurs whenever an object tries to use the RenderedImage data or tries to find out information regarding some of the RenderedImage's metadata, such as image width or image height. Another way a rendering can be performed is by using the createRendering method. This method creates a rendering without marking the RenderedOp node as being rendered. The importance of this classification will be described in the “RenderingChangeEvents” section. Before this can be discussed, we first need to examine the different parts of a RenderedOp: the operation, the parameter block, and the rendering hints.

Operations

With respect to RenderedOps, an operation is simply a String specifying how to create a destination RenderedImage. Some examples of valid operations are add, addConst, and invert, where the first operation adds two source images, the second operation adds a source image to an array of constants (one constant per image band), and the last operation inverts a source image.

Each allowable operator corresponds to a class implementing the javax.media.jai.OperationDescriptor interface. Each of these classes describes how their corresponding operation works. They also describe the number of source objects and the number and types of parameters they require. For example, operator descriptor classes for the previously listed operations are javax.media.jai.AddDescriptor, javax.media.jai.AddConstDescriptor, and javax.media.jai.InvertDescriptor. OperationDescriptor classes will be covered more completely in the later section entitled “Extending JAI.”

Another class that will be discussed in the “Extending JAI” section is the javax.media.jai.OperationRegistry. All allowable operations must be registered in order for them to be used. In Listing 6.4, the OperationRegistry object is used to display the set of registered operations.

Listing 6.4 ListRegistry.java

package ch6;

import javax.media.jai.JAI;
import javax.media.jai.OperationRegistry;
import javax.media.jai.RegistryMode;

/**
   lists all allowable JAI operations
 */
public class ListRegistry {
    public ListRegistry() {
        or = JAI.getDefaultInstance().getOperationRegistry();
        String[] modeNames = RegistryMode.getModeNames();
        String[] descriptorNames;

        for (int i=0;i<modeNames.length;i++) {
            System.out.println("For registry mode: " + modeNames[i]);

            descriptorNames = or.getDescriptorNames(modeNames[i]);
            for (int j=0;j<descriptorNames.length;j++) {
                System.out.print("	Registered Operator: ");
                System.out.println(descriptorNames[j]);
            }
        }
    }

    public static void main(String[] args) {
        new ListRegistry();
    }

    private OperationRegistry or;
}
							

ParameterBlock

The java.awt.image.renderable.ParameterBlock class is used to encapsulate information regarding sources and parameters necessary for a particular operation to be carried out. For example, for an add operation, the corresponding ParameterBlock would need to contain two sources. For the addConst operation, it would need to contain one source and one array of constants (one for each band), and for the invert operation, a single source is all that is required. To place a PlanarImage source in a ParameterBlock one can use the ParameterBlock's addSource method, and to place a parameter in a ParameterBlock one can use its add method. For example, a ParameterBlock that could be used for an addConst operation is as follows:

ParameterBlock pb = new ParameterBlock()
pb.addSource(planarImageSource);
pb.add(constantDoubleArray);

One last note regarding ParameterBlocks is that once a ParameterBlock is created it can be changed using one of the ParameterBlock's set or setSource methods. For example, to reuse the preceding ParameterBlock to perform an addConst operation on another source image, simply use

setSource(newPlanarImageSource, 0); //where 0 refers to the source index

As will be discussed in the “RenderingChangeEvents” section, changing a ParameterBlock contained within a RenderedOp causes the parts of the DAG dependent on that ParameterBlock to change. So by simply changing a contained ParameterBlock, not only will its associated operation be carried out again, but all operations dependent on the created RenderedImage will be redone.

RenderingHints

As described in Chapter 3, “Graphics Programming with the Java 2D API,” and Chapter 4, “The Immediate Mode Imaging Model”, the java.awt.RenderingHints class provides hints for use when creating a RenderedImage. All these hints have default values, so a null can be used whenever a RenderingHints object is expected. These default values are considered the global set of rendering hints. Whenever a RenderedOp is created using the JAI's create method, a non-null or local set of rendering hints can be passed to it in order to override one or more global rendering hints.

At the end of this chapter, when we discuss RenderableImages, you will see that a single set of local rendering hints can be provided to the final RenderableOp in a RenderableOp DAG, and that set will be combined with the global set and used for all operations in that DAG. This is unlike a RenderedOp DAG in which every node can have its own set of local rendering hints.

RenderingChangeEvents

When a RenderedOp is rendered using the getRendering method (either implicitly or explicitly) it is marked as being rendered. After this occurs, any time it gets rerendered it sends out a javax.media.jai.RenderingChangeEvent event to any object that had registered itself as being interested in these events. Because the RenderingChangeEvent is a subclass of the java.beans.PropertyChangeEvent, any object interested in receiving RenderingChangeEvents can implement the PropertyChangeListener interface and register themselves as such using the PlanarImage's addPropertyListener method.

An interesting aspect of the RenderedOp class is that whenever a RenderedOp node is created, it registers itself as a PropertyChangeListener for all of its immediate source nodes. This way, whenever one of these source nodes gets rerendered, it can also rerender itself. In the same manner, if any node in the RenderedOp DAG gets rerendered, all the following RenderedOp nodes will rerender themselves. The question now is: what can make a RenderedOp node rerender itself in order to start this process? The answer is any change in its operation, or its ParameterBlock object or its RenderingHints object. Thus, if in Figure 6.5, you change the filename contained in a ParameterBlock in one of the top RenderedOp nodes, that RenderedOp and all the dependent RenderOp nodes will rerender their images, causing the final RenderedImage to change.

An example of this situation appears in Listing 6.5. In this listing, a PlanarImage is created, rotated, and displayed. The name of the file to be loaded is then changed in the initial ParameterBlock, causing the corresponding RenderedOp to rerender its image. This RenderedOp sends out a RenderingChangeEvent so the next RenderedOp also rerenders its image. When run, Listing 6.5 displays the image corresponding to the first filename adjacent to a rotated version of this image. After a delay of two seconds, both images change to depict the change in the name of the file to be loaded.

Listing 6.5 RenderingChangeEventTest.java

package ch6;

import java.awt.*;
import java.io.*;
import java.util.*;
import javax.swing.*;
import java.awt.image.renderable.ParameterBlock;
import java.awt.image.RenderedImage;
import javax.media.jai.JAI;
import javax.media.jai.PlanarImage;
import javax.media.jai.RenderedOp;

/**
   RenderingChangeEventTest.java -- objects of this class
   1.  create a RenderedOp by loading filename1
   2.  create a 2nd RenderedOp representing rotation
       of the first RenderedOp
   3.  renders and displays both RenderedOps
   4.  changes the input filename used in the 1st RenderedOp
       to filename2 which generates a RenderingChangeEvent causing
       the rotated image to change.
*/
public class RenderingChangeEventTest extends JFrame {

    public RenderingChangeEventTest(String filename1, String filename2) {
        this.filename2 = filename2;
        RenderedOp inputRO = JAI.create("fileload", filename1);
        RenderedOp rotatedRO = createRotatedImage(inputRO);

        /*
          Force rotatedRO to be rendered.  This is not usually needed,
          but for this example we need the rotatedRO object's rendering
          to be done before the rotation angle is changed.
        */
        RenderedImage tmp = (PlanarImage)rotatedRO.getRendering();

        // display original and rotated
        getContentPane().setLayout(new GridLayout(1,2));
        getContentPane().add(new ch6Display(inputRO));
        getContentPane().add(new ch6Display(rotatedRO));
        pack();
        show();

        // wait 2 seconds so images don't change to quickly
        try{
            Thread.sleep(2000);
        }
        catch(InterruptedException ie) {
        }

        changeFilename(rotatedRO);

        // redisplay images
        repaint();
    }

    /**
       Returns a RenderedOp representing a rotated
       version of RenderedOp toBeRotatedRO
    */
    private RenderedOp createRotatedImage(RenderedOp toBeRotatedRO) {
        float angle = (float)((45.0/180.0)*Math.PI); //45 degree rotation

        ParameterBlock param;
        param =  new ParameterBlock();
        param.addSource(toBeRotatedRO);
        param.add(new Float(toBeRotatedRO.getWidth()/2));
        param.add(new Float(toBeRotatedRO.getHeight()/2));
        param.add(new Float(angle));
        RenderedOp ro = JAI.create("rotate", param);

        return ro;
    }

    /**
       1.  go to the RenderedOp's source image
       2.  change the filename parameter in its ParameterBlock
       3.  this will generate a RenderingChangeEvent which will
       cause the rotatedRO RenderedOp to rerender its images
    */
    private void changeFilename(RenderedOp toBeChangedRO) {
        //get source RenderedOp
        RenderedOp tmpRO = (RenderedOp)toBeChangedRO.getSourceImage(0);
        tmpRO.setParameter(filename2, 0);
    }

    public static void main(String[] args) {
        if (args.length != 2)
            System.err.println("Usage:  filename1 filename2");
        else
            new RenderingChangeEventTest(args[0], args[1]);
    }
    String filename2;
}