Inside the Rendering Pipeline
As we have said, 3D graphics involves creating a 3D model, establishing a viewing volume, and then rendering that viewing volume as a 2D plane. A pretty standard series of steps is used in creating the pixels that are ultimately put on the screen. This series of steps is frequently referred to as the rendering pipeline. The vast majority of 3D APIs follow these steps (shown in Figure 10.6). The top row refers to the five basic steps in the pipeline with the arrow indicating the order of steps. The bottom row indicates roughly where Java 3D fits into this picture. Note, however, that major differences exist in what specifically happens at each step.
Figure 10.6. Schematic of the typical rendering pipeline.
You will gain some insight into the steps of the pipeline shortly, but for now we want to consider the role of Java 3D in this pipeline.
Because many of the steps in 3D graphics are so computationally expensive, they are often implemented in optimized native code. Most current 3D graphics cards have incorporated hardware acceleration to perform most of these low-level API calls. The two most prominent low-level 3D graphics APIs are OpenGL—which runs on Sun, SGI, Linux, and Windows—and DirectX, which runs on Windows machines. As yet, no hardware acceleration works directly with Java 3D (nor is there ever likely to be), and therefore Java 3D makes use of bindings to the low-level API (either DirectX or OpenGL) through separate versions of Java 3D for OpenGL and DirectX.
It is a mistake, however, to believe that Java 3D is simply a set of bindings to the low-level API. Because Java 3D is written in Java, it turns out that all low-level calls have to be made through the Java Native Interface (JNI). Each call to the JNI is expensive, so the calls need to be used as sparingly as possible and scheduled appropriately. Therefore, Java 3D substantially reduces the computational problem within its own native rendering layer before calling the low-level API. This special rendering layer is the heart of the Java 3D renderer.
The low-level bindings that are the end product of the Java 3D renderer are largely invisible to the programmer, and for some applications this can be a problem. Future implementations promise to allow the programmer to make calls to the low-level API, but at present we must accept the fact that we cannot easily make low-level API calls. In general, this isn't a major problem, and unless a programmer is very good, there won't be much advantage to making these calls anyway. In the vast majority of the cases, Java 3D will be sufficient to the task without needing to get into low-level calls.
To reiterate, a developer might consider directly using either of the low-level APIs (OpenGL or DirectX) in order to get performance improvements. This brute force approach has been used successfully many times; however, for a huge majority of applications, this isn't a wise choice. Performance improvements are predicated by whether the programmer is good enough or experienced enough to beat Java 3D. Even if the programmer operating at a low level can do a better job of squeezing performance out of the application (again, by no means guaranteed), there will be a loss of the cross-platform capabilities (particularly with DirectX) and the development cycle will almost always be much longer.
The Framebuffer
Probably the most important innovation and concept to understand regarding raster graphics hardware is the idea of a framebuffer. The framebuffer is quite similar to the image buffer idea presented in Part I, “2D Graphics and Imaging on the Java Platform: The Java 2D, Java Advanced Imaging, and Java Image I/O APIs.” Essentially, we are talking about a memory space to store pixel information.
The framebuffer in 3D graphics is more typically a set of buffers that store different types of information. The most commonly considered buffer (and the one used in the imaging section) is the color buffer. In the simplest case, the color buffer contains RGB plus alpha values for each pixel in the output image.
The second important component of the framebuffer that should be understood is the depth buffer. We will cover the depth buffer in more detail when we discuss hidden surface removal in the section “Reducing Unnecessary Rendering Through Culling.” For now, understand that the depth buffer stores information about the distance of objects in the 3D view volume from the eye of the person doing the viewing. This information is later collapsed such that the closest object obscures the objects behind it.
Other buffers such as the accumulation buffer and the stencil buffer are less commonly used by application programmers and therefore aren't covered here. Both the color buffer and depth buffer will play a role in the discussions that follow.
Rasterization
We discussed the process of rasterization for 2D graphics in Chapter 2, “Imaging and Graphics on the Java Platform,” under the section “What Is Rendering?” Rasterization is the conversion of the mathematical description of a primitive with its color information into screen coordinates. Extension of the 2D concept of rasterization to the case of 3D graphics is pretty straightforward with the exception of a few additional steps dealing with depth, which are covered next.
Java 3D Rendering Modes
Given an understanding of the general rendering process, we can now examine some aspects of the Java 3D rendering process.
Java 3D has four primary rendering modes that allow differing amounts of control over the low-level aspects of rendering. Generally, when the user has more control over rendering, Java 3D will be able to perform fewer optimizations. Java 3D's four rendering modes are listed in Table 10.1.
The choice of rendering mode is usually quite easy: Stick with the retained or compiled-retained mode unless strongly compelled to do otherwise. Before discussing why we make this statement, a quick overview of the four rendering modes is presented.
Retained Mode
The retained mode is the default mode. In retained mode, the application remains in a continuous rendering state unless the Canvas3D.stopRender() method is invoked. Retained mode represents a happy medium between total capitulation to the Java 3D renderer with compiled-retained mode (covered next) and near total abandonment of the Java 3D renderer with the pure immediate mode (also covered next). The optimization that is automatically generated in the retained mode is substantial. In particular, the Java 3D renderer will attempt to build special structures for geometry handling and flatten transform operations. (Transforms are covered later in this chapter and the next. )
In many cases, picking objects in the 3D scene is most easily solved in retained mode because the geometry compression that occurs with compiling (discussed next) can interfere with some operations.
Compiled-retained Mode
Compiled-retained mode gives Java 3D the freedom to perform the greatest number of optimizations. Important among these optimizations are scene graph compression and geometry compression and grouping.
To enter compiled-retained mode, the programmer calls the BranchGroup's compile() method. The compile() method optimizes the scene graph in two basic ways; first by sorting the attributes, and second by grouping shapes.
The disappointing truth is that you aren't likely to see much of an improvement in frame rate after compiling. This is for two primary reasons. The first reason is that most of the default capability bits aren't set up to allow for optimization of the scene graph. This is easily corrected by setting these bits at instantiation (covered in more detail in the following chapter; also see the previous sidebar). Manually setting the capability bits like this might be a lot of working for little gain.
The second reason that you might not see an increase in speed by compiling is that even if all the capability bits are set correctly, there might not be enough shapes to make much of a difference anyway. The main additional optimization that you get from compiling is geometry compression, and this is only useful with complex shapes. The other optimization that compiled-retained mode has over retained mode is geometry grouping, which is only an advantage when large numbers of shapes are part of the scene. In these cases, you might indeed be able to detect a difference by compiling.
Immediate Mode
The immediate mode, in contrast to the two modes previously described, provides a lot of runtime flexibility. This comes at the cost of severely limiting the kinds of optimizations that the Java 3D renderer can use.
Generally, a programmer who uses immediate mode is concerned with controlling rendering at the lowest possible level (that is, at the level of the frame buffer). Immediate-mode programmers often just want to take advantage of the geometry classes and to leave the other stuff behind. Their choice in using the immediate mode is often one of vanity rather than sanity because the optimizations that occur in retained or compiled-retained mode (for example, geometry compression, geometry grouping, scene graph flattening) are quite powerful. That said, Java 3D does provide a fairly robust immediate-mode rendering model and can accommodate most of the needs of immediate mode users. While in immediate mode, the Java 3D renderer is off and all rendering is done by direct calls. You should note that many of the rendering efficiencies built in an immediate-mode application on one platform won't necessarily result in the same efficiencies on a different platform.
To continue, the real rendering bottleneck is usually in scanning conversion and shading the triangles—not in the process of looping through frames. The Java 3D renderer helps us to avoid rendering unnecessary triangles, which, in most cases, is where the bulk of the optimizations are beneficial.
Mixed-mode Rendering
Finally, it should be noted that the immediate mode can be mixed with retained and compiled-retained modes. This option is often chosen in practice when low-level access to the frame buffer is needed. The mixed mode retains the property of continuous rendering from the retained and compile-retained modes. Four methods from the Canvas3D class (see the next chapter) can be overridden to allow the application to access the buffer at different stages of the pipeline. These methods are described in Table 10.2.