The First Steps
In this chapter we deal with the first practical steps that will accompany us for the rest of the book.
First we will show how to set up our first working rendering inside an HTML page by using WebGL. More precisely, we will draw a triangle. We advise you that at first it would look like you need to learn an awful lot of information for such a simple task as drawing a triangle, but what we will do in this simple example is the same as what we would do for a huge project. Simply put, you need to learn to drive a car for going 10 miles as well as 100 miles.
Then we will introduce the EnvyMyCar (NVMC) framework that we will use for the rest of this book for putting into use the theory we will learn along the way. Briefly, NVMC is a simple car racing game where the gamer controls a car moving on a track. What makes NVMC different from a classic video game is that there is no graphics at all, because we are in charge to develop it. Given a complete description of the scene (where is the car, how is it oriented, what is its shape, how is the track made, etc.) we will learn, here and during the rest of the book, how to draw the elements and the effects we need to obtain, at the end, a visually pleasant car racing game.
2.1 The Application Programming Interface
The rasterization-based pipeline is one of several possible graphics architectures. As we have seen in Section 1.3.2, the logical pipeline is composed of a series of stages, through which data coming from the application is manipulated to obtain a raster image. The logical pipeline will be our desk bench to explain the theory behind graphics structures and techniques.
As it happens for many computer science fields, a particular technology is characterized by having a common substructure. Over this substructure, several interaction protocols are defined and, most of the time, standardized. Whoever wants to comply with these protocols must respect them. In our context, the technology we are dealing with is the one implemented by computer graphics systems, which may include software components or hardware chips, and the protocol is the Application Programming Interface, or API. The API defines the syntax of constant values, data structures and functions (e.g., symbolic names and function signatures), and, more importantly, their semantics (e.g., what they mean and what they do). The graphics API will be our tool to interact with the graphics system and write computer graphics applications.
To clarify these concepts, let us make an example: the car. The Authority specifies how cars have to be made and how they must behave. A car must have wheels, a steering wheel, pedals, levers, and so on. It also specifies what happens if a certain pedal is pushed (e.g., accelerator or brake), or a certain lever is moved (e.g., indicators). A car manufacturer has to follow these instructions to build a vehicle that may be approved by the Authority. Knowing the specifications, a car driver acts on pedals and levers in a certain way to reach his destination. In this example, the Authority defines the car specifications, the manufacturer implements them, and the driver uses them.
For the time being, the most used graphics APIs for real-time rendering are DirectX [42], and OpenGL [17]. Considering OpenGL in the car example, the Authority is the Khronos Groups [19], the manufacturers are NVidia, ATI, or Intel, and we are the drivers.
The original OpenGL API targeted desktops to supercomputers. Given its adoption in academic and industrial contexts, derivatives have been proposed, like OpenGL|SC for safety-critical applications, and OpenGL|ES for embedded systems with lower capabilities with respect to home PCs. The API specifications are language agnostic (that is, they can be implemented in any modern programming language), but are mostly designed to be executed in an unrestricted environment (that is, native executable binary code). This means that using OpenGL in a strictly controlled platform, such as the JavaScript virtual machine of modern Web browsers, is not possible without imposing some additional restrictions.
The OpenGL specifications are defined by the Khronos Group, a third party implements the specifications and provides the API, and a graphics programmer uses it to create graphics applications or libraries. From a programmer’s point of view, it is not important how the third party actually implements the API (e.g., in the car example, whether the manufacturer uses an oil or an electric engine), as long as it strictly follows the specifications. This means that OpenGL implementations can be both a pure software library, or a system driver that communicates with a hardware device (e.g., a graphics accelerator).
To allow web pages to take advantage of the host system graphics capabilities, the Khronos Group standardized the WebGL graphics API [18]. WebGL is a graphics API written to the OpenGL|ES 2.0 specifications that allows accessing the graphics system, that is, the hardware graphics chip, from within standard HTML pages, with some minor restrictions introduced to address security issues.
As concepts are being developed, we want a comfortable platform that lets us practice on what we learn with as few burdens as possible. For this reason, we will use the WebGL for our code examples. By using the HTML, JavaScript, and WebGL standard technologies we do not have to install and use ad-hoc programming tools. We just need a Web browser that supports WebGL and a text editor, and we will be able to test, run and deploy the code we write on a wide range of devices, from powerful personal computers to smartphones.
2.2 The WebGL Rasterization-Based Pipeline
The WebGL pipeline is the concrete graphics pipeline that we will use throughout this book. Before going into our first rendering example, we must have a big picture of how it works. Figure 2.1 depicts the stages that compose the WebGL pipeline, and which are the entities they involve. The application communicates with the graphics system (e.g., the graphics hardware or the software library) with the WebGL API. When drawing commands are issued, data starts to flow from graphics memory and gets consumed or transformed by each pipeline stage, which we will discuss in the following:
Vertex Puller (VP). The purpose of the vertex puller stage is simply to fetch data associated to vertex attributes from graphics memory, pack them and pass them down to the next stage (the vertex shader). The vertex puller represents the first stage of the geometry pipeline, and can manage a maximum fixed number of vertex attributes that depends on the WebGL implementation. Each attribute is identified by an attribute index, and only the attributes needed by the vertex shader are actually fetched. For each attribute, the vertex puller must be instructed about where and how the associated data has to be fetched from graphics memory. Once all attributes are fetched, all together they represent a raw data bundle, namely the pipeline input vertex. After passing the vertex to the next stage, the process starts again and continues until all required vertices have been assembled and forwarded. For example, we could configure the vertex puller such that the data associated to attribute 0 is a constant four-dimensional value, and the data of attribute 3 is a two-dimensional value that must be fetched from graphics memory, starting at a particular address.
Vertex Shader (VS). The attributes of the input vertex assembled by the vertex puller arrive at the vertex shader, where they will be used to produce new attributes forming the transformed vertex. This process is carried out by a user-defined procedure, written in the OpenGL Shading Language (GLSL). This procedure is referred to as a vertex shader, just like the pipeline stage name. A vertex shader takes as input n general attributes, coming from the VP, and gives as output m general attributes, plus a special one, namely the vertex position. The resulting transformed vertex is passed to the next stage (primitive assembler). For example, to visualize the temperature of the Earth’s surface, a vertex shader could receive in input three scalar attributes, representing a temperature, a latitude and a longitude, and output two 3D attributes, representing an RGB color and a position in space.
Primitive Assembler (PA). The WebGL rasterization pipeline can only draw three basic geometric primitives: points, line segments, and triangles. The primitive assembler, as the name suggests, is in charge of collecting the adequate number of vertices coming from the VS, assembling them in a t-uple, and passing it to the next stage (the rasterizer). The number of vertices (t) depends on the primitive being drawn: 1 for points, 2 for line segments, and 3 for triangles. When an API draw command is issued, we specify which is the primitive that we want the pipeline to draw, and thus configure the primitive assembler.
Rasterizer (RS). The rasterizer stage receives as input a primitive consisting of t transformed vertices, and calculates which are the pixels it covers. It uses a special vertex attribute that represents the vertex position to identify the covered region, then, for each pixel, it interpolates the m attributes of each vertex and creates a packet, called fragment, containing the associated pixel position and the m interpolated values. Each assembled fragment is then sent to the next stage (the Fragment Shader). For example, if we draw a segment whose vertices have an associated color attribute, e.g., one red and one green, the rasterizer will generate several fragments that, altogether, resemble a line: fragments near the first vertex will have associated a reddish color that becomes yellow at the segment midpoint, and then goes to green while approaching the second vertex.
Fragment Shader (FS). Similarly to the vertex shader, the fragment shader runs a user-defined GLSL procedure that receives as input a fragment with a read-only position Fxy and m attributes, and uses them to compute the color of the output pixel at location Fxy. For example, given two input attributes representing a color and a darkening factor, the output color could be the darkened negative of the input color.
Output Combiner (OC). The last stage of the geometry pipeline is the output combiner. Before writing to the framebuffer the pixels coming out from the fragment shader, the output combiner executes a series of configurable tests that can depend both on the incoming pixel data, and the data already present in the framebuffer at the same pixel location. For example, an incoming pixel could be discarded (e.g., not written) if it is not visible. Moreover, after the tests have been performed, the actual color written can be further modified by blending the incoming value with the existing one at the same location.
Framebuffer Operations (FO). A special component of the rendering architecture is dedicated to directly access the framebuffer. The frame-buffer operations component is not part of the geometry pipeline, and it is used to clear the framebuffer with a particular color, and to read back the framebuffer content (e.g., its pixels).
All the stages of the WebGL pipeline can be configured by using the corresponding API functions. Moreover, the VS and FS stages are programmable, e.g., we write programs that they will execute on their inputs. For this reason, such a system is often referred to as a programmable pipeline, in contrast to a fixed-function pipeline that does not allow the execution of custom code.
2.3 Programming the Rendering Pipeline: Your First Rendering
Throughout this book we will use simple HTML pages as containers for showing our computer-generated images and for handling general user interface controls. Moreover, we will use JavaScript as our programming language because it is both the primary scripting language to be natively integrated in HTML, and it is the language against which the WebGL specification is written. To this extent, the reader is required to have a well-founded knowledge in general programming, and basic notions of HTML and JavaScript.
In this first practical exercise we will write a very simple HTML page that displays the most basic polygonal primitive, a triangle, using JavaScript and WebGL. We subdivide our goal into the following main steps:
- define the HTML page that will display our drawing
- initialize WebGL
- define what to draw
- define how to draw
- perform the actual drawing
Steps 3 and 4 do not have inter-dependencies, so their order can be exchanged. Even if this is the first rendering example, it will expose some of the fundamental concepts used throughout this book and that will be expanded as new theoretical knowledge is acquired. In the following we will implement the above steps in a top-down fashion, meaning that we will refine our code as steps are examined.
Step 1: The HTML Page
The first thing to do is to define the HTML page that we will use to display our rendering:
1 <html>
2 <head>
3 <script type="text/javascript">
4 // ... draw code here ...
5 </script >
6 </head>
7 <body>
8 <canvas
9 id = "OUTPUT-CANVAS"
10 width = "500px"
11 height = "500px"
12 style = "border: 1px solid black"
13 ></canvas >
14 </body>
15 </html>
LISTING 2.1: HTML page for running the client.
As just stated we assume that the reader is familiar with basic HTML. In brief, the html root tag (lines 1 to 15) encapsulates the whole page; it is the container of two basic sections, the head section (lines 2 to 6) that contains metadata and scripts, and the body section (lines 7 to 14) that contains the elements shown to the user. A fundamental element of the page is the canvas tag on lines 8 to 13. Introduced in the HTML5 standard, the HTMLCanvasElement represents, as its name suggests, a (rectangular) region of the page that can be used as the target for drawing commands. Like a painter who uses brushes and colors to draw on his or her canvas, we will use WebGL through JavaScript to set the color of the pixels inside our output region. In the HTML code, we also define the id, width and height attributes to set, respectively, the canvas identifier, width and height. With the style attribute we also set up a 1-pixel-wide black border to help us visualize the rectangular region occupied by the canvas inside the page. From now on, our job is to write the JavaScript code inside the script tag, using the code in Listing 2.1 as our base skeleton.
1 // <script type="text/javascript">
2 // global variables
3 // ...
4
5 function setupWebGL () {/* ... */}
6 function setupWhatToDraw () {/* ... */}
7 function setupHowToDraw () {/* ... */}
8 function draw () {/* ... */}
9
10 function helloDraw () {
11 setupWebGL();
12 setupWhatToDraw();
13 setupHowToDraw();
14 draw() ;
15}
16
17 window.onload = helloDraw;
18 // </script >
LISTING 2.2: Skeleton code.
In this example it is important that the code we write will not be executed before the page loading has completed. Otherwise, we would not be able to access the canvas with document.getElementById() simply because the canvas tag has not been parsed yet and thus could not be queried. For this reason we must be notified by the browser whenever the page is ready; by exploiting the native and widely pervasive use of object events in a Web environment, we accomplish our task by simply registering the helloDraw function as the page load event handler, as shown on line 17 in Listing 2.2.
Step 2: Initialize WebGL
As preparatory knowledge, we need to understand how to interact with the WebGL API. In OpenGL and all its derivatives, the graphics pipeline works as a state machine : the outcome of every operation (in our case, every API function call) is determined by the internal state of the machine. The actual OpenGL machine state is referred to as the rendering context, or simply context. To help handle this concept more practically, we can think of a context as a car and its state as the state of the car, that is, its position, velocity acceleration, the rotation of the steering wheel, the position of the pedals. The effect of the actions we perform depends on the state: for example the effect we obtain rotating the steering wheel is different if the car is moving or not.
When using OpenGL or OpenGL|ES, once a context is created and activated, it becomes syntactically hidden to the API: this means that every function call acts implicitly on the currently active context, which thus needs not to be passed as argument in any of them. WebGL makes the context explicitly available to the programmer, encapsulating it in a JavaScript object with a specific interface, the WebGLRenderingContext: using WebGL thus means creating a context object and then interacting with it by calling its methods.
Every WebGLRenderingContext object is tied to an HTMLCanvasElement that it will use as the output of its rendering commands. The creation of a context is accomplished by requesting it to the canvas, as shown in Listing 2.3.
1 // global variables
2 var gl = null; // the rendering context
3
4 function setupWebGL() {
5 var canvas = document.getElementById("OUTPUT-CANVAS");
6 gl = canvas.getContext("webgl");
7}
LISTING 2.3: Setting up WebG1.
The first thing to do is to obtain a reference to the HTMLCanvasElement object: this is done on line 5, where we ask the global document object to retrieve an element whose identifier (its id) is OUTPUT-CANVAS, using the method getElementById. As you can argue, the canvas variable is now referencing the canvas element on line 8 in Listing 2.1. Usually, the canvas will provide the rendering context with a framebuffer that contains a color buffer consisting of four 8-bit channels, namely RGBA, plus a depth buffer whose precision would be of 16 to 24 bytes, depending on the host device. It is important to say that the alpha channel of the color buffer will be used by the browser as a transparency factor, meaning that the colors written when using WebGL will be overlaid on the page in compliance with the HTML specifications.
Now we are ready to create a WebGLRenderingContext. The method get-Context of the canvas object, invoked with the string webgl as its single argument, creates and returns the WebGL context that we will use for rendering. For some browsers, the string webgl has to be replaced with experimental-webgl. Note that there is only one context associated with each canvas: the first invocation of getContext on a canvas causes the context object to be created and returned; every other invocation will simply return the same object. On line 6 we store the created context to the gl variable. Unless otherwise specified, throughout the code in this book the identifier gl will be always and only used for a variable referencing a WebGL rendering context.
Step 3: Define What To Draw
It is important to note that the WebGL specifications, along with other rasterization-based graphics API, are designed to take into account and efficiently exploit the graphics hardware that we find on our devices, from smart-phones to powerful personal computers. One of the most important practical effects of the design is that common data structures a programmer would use to describe some entities must be mirrored with their corresponding counterpart in the API. That is, before using them, we have to encapsulate their data in an appropriate WebGL structure.
For reasons that we will soon explain, we treat our 500 × 500 pixels canvas as a region that spans the plane from −1 to 1 both horizontally and vertically, instead of 0 to 499. This means that, in the hypothetical unit of measure we used, the canvas is two units wide and two units tall. As a first example, let’s consider the triangle in Figure 2.2 (Top-Left): it is composed by three vertices on the XY plane whose coordinates are (0.0,0.0), (1.0, 0.0) and (0.0,1.0). In JavaScript, a straightforward way to express this is indicated in Listing 2.4:
1 var triangle = {
2 vertexPositions : [
3 [0.0, 0.0], // 1st vertex
4 [1.0, 0.0], // 2nd vertex
5 [0.0, 1.0] // 3rd vertex
6]
7};
LISTING 2.4: A triangle in JavaScript.
Illustration of the mirroring of arrays from the system memory, where they can be accessed with JavaScript, to the graphics memory.
The triangle variable refers to an object with a single property named vertexPositions. In turn, vertexPositions refers to an array of three elements, one for each vertex. Each element stores the x and y coordinates of a vertex with an array of two numbers. Although the above representation is clear from a design point of view, it is not very compact in terms of occupied space and data access pattern. To achieve the best performance, we must represent the triangle in a more raw way, as shown in Listing 2.5.
1 var positions = [
2 0.0, 0.0, // 1st vertex
3 1.0, 0.0, // 2nd vertex
4 0.0, 1.0 // 3rd vertex
5] ;
LISTING 2.5: A triangle represented with an array of scalars.
As you can notice, the triangle is now represented with a single array of six numbers, where each number pair represents the two-dimensional coordinates of a vertex. Nonetheless, storing the attributes of the vertices that compose a geometric primitive (that is, the positions of the three vertices that form a triangle as in the above example) in a single array of numbers, coordinate after coordinate, and vertex after vertex, is exactly the way WebGL requires us to follow whenever geometric data has to be defined.
Now we have to take a further step to convert the data in a lower level representation. Since JavaScript arrays do not represent a contiguous chunk of memory and, moreover, are not homogeneous (e.g., elements can have different types), they cannot be directly delivered to WebGL, which expects a raw, contiguous region of memory. For this reason, the WebGL specifications lead the way to the definition of new JavaScript objects for representing contiguous and strongly-typed arrays. The typed array specification defines a series of such objects, i.e., Uint8Array (unsigned, 8-bit integers) and Float32Array (32-bit floating-points), that we will use for creating the low-level version of our native JavaScript array. The following code constructs a 32-bit floatingpoint typed array from a native array:
1 var typedPositions = new Float32Array(positions);
Alternatively, we could have filled the typed array directly, without passing by a native array:
1 var typedPositions = new Float32Array(6); // 6 floats
2 typedPositions [0] = 0.0; typedPositions [1] = 0.0;
3 typedPositions [2] = 1.0; typedPositions [3] = 0.0;
4 typedPositions [4] = 0.0; typedPositions [5] = 1.0;
The data, laid out as above, is now ready to be mirrored (for example by creating an internal WebGL copy) and encapsulated in a WebGL object. As mentioned above, WebGL uses its own counterpart of a native data structure: in this case, a JavaScript typed array containing vertex attributes is mirrored by a WebGLBuffer object:
1 var positionsBuffer = gl.createBuffer();
2 gl.bindBuffer(gl.ARRAY_BUFFER, positionsBuffer);
3 gl.bufferData(gl.ARRAY_BUFFER, typedPositions, gl.STATIC_DRAW);
On line 1 an uninitialized, zero-sized WebGLBuffer is created and a reference to it is stored in the positionsBuffer variable. On line 2 we tell WebGL to bind positionsBuffer to the ARRAY_BUFFER target. This is the first example where the state machine nature of WebGL emerges: once an object O has been bound to a particular target T, every operation addressing T will operate on O. This is actually what happens on line 3: we send the data contained in the typed array typedPositions to the object that is bound on target ARRAY_BUFFER, that is, the WebGLBuffer positionsBuffer. The ARRAY_BUFFER target is specific for buffers that store vertex attributes. The third parameter of the method bufferData is a hint to the WebGL context that informs it how we are going to use the buffer: by specifying STATIC_DRAW we declare that we are likely to specify the buffer content once but use it many times.
As can be deduced from the above low-level code, from the point of view of the WebGL rendering pipeline a vertex is nothing but a set of vertex attributes. In turn, a vertex attribute is a scalar value (e.g., a number), or a two-, three- or four-dimensional vector. The allowed scalar type for both numbers and vectors are integers, fixed- and floating-point values. The WebGL specifications impose restrictions on the number of bits used by each representation. Deciding the most adequate data format for a vertex attribute can be important for both quality and performances, and it has to be evaluated depending on the application and on the software and hardware resources at disposal.
As we have seen when we prepared the buffer, for a WebGL programmer a vertex attribute is a small chunk of memory that contains one to four numbers, and a vertex is formed by a set of attributes. The next step is to tell the context how to fetch the data of each vertex attribute (in our example, only the two-dimensional position attribute). The first operation executed by the pipeline is to gather all the attributes a vertex is composed of, then pass this data bundle to the vertex processing stage. As illustrated in Figure 2.3 (on the left), there are a number of attribute slots we can use to compose our vertex (the actual number of slots is implementation dependent).
The next step is to select a slot and tell the context how to fetch data from it. This is accomplished with the following code:
1 var positionAttriblndex = 0;
2 gl.enableVertexAttribArray(positionAttriblndex);
3 gl.vertexAttribPointer(positionAttribIndex , 2, gl.FLOAT, false, 0, 0);
At line 1, we store the index of the selected slot in a global variable, to be used later. Selecting index zero is completely arbitrary: we can choose whichever indices we want (in case of multiple attributes). At line 2, with the method enableVertexAttribArray we tell the context that vertex attribute from slot positionAttriblndex (zero) has to be fetched from an array of values, meaning that we will latch a vertex buffer as the attribute data source. At last, we must specify the context, which is the data type for the attribute and how to fetch it. The method vertexAttribPointer at line 3 solves this purpose; using a C-like syntax, its prototype is:
1 void vertexAttribPointer(unsigned int index, int size, int type,
2 bool normalized, unsigned int stride, unsigned int offset);
The index parameter is the attribute index that is being specified; size represents the dimensionality of the attribute (two-dimensional vector); type is a symbolic constant indicating the attribute scalar type (gl.FLOAT, e.g., floating point number); normalized is a flag indicating whether an attribute with integral scalar type must be normalized (more on this later, at any rate, the value here is ignored because the attribute scalar type is not an integer type); stride is the number of bytes from the beginning of an item in the vertex attribute stream and the beginning of the next entry in the stream (zero means that there are no gaps, that is the attribute is tightly packed, three floats one after another); offset is the offset in bytes from the beginning of the WebGLBuffer currently bound to the ARRAY_BUFFER target (in our case, positionsBuffer) to the beginning of the first attribute in the array (zero means that our position starts immediately at the beginning of the memory buffer).
The complete shape setup code is resembled in Listing 2.6.
1 // global variables
2 // ...
3 var positionAttribIndex = 0;
4
5 function setupWhatToDraw() {
6 var positions = [
7 0.0, 0.0, // 1st vertex
8 1.0, 0.0, // 2nd vertex
9 0.0, 1.0 // 3rd vertex
10];
11
12 var typedPositions = new Float32Array(positions);
13
14 var positionsBuffer = gl.createBuffer ();
15 gl.bindBuffer(gl.ARRAY_BUFFER, positionsBuffer);
16 gl.bufferData(gl.ARRAY_BUFFER, typedPositions,gl.STATIC_DRAW);
17
18 gl.enableVertexAttribArray(positionAttribIndex);
19 gl.vertexAttribPointer(positionAttribIndex, 2, gl.FLOAT, falser, 0, 0);
20}
LISTING 2.6: Complete code to set up a triangle.
The rendering context is now configured to feed the pipeline with the stream of vertex attributes we have just set up.
Step 3: Define How to Draw
Once the pipeline has been configured to fetch the triangle vertices, we have to specify the operations we want to execute on each vertex. As just described in Section 2.2, the vertex shader stage of the WebGL pipeline corresponds to the per-vertex operations stage of the logical pipeline (seen in Section 1.3.2). In this stage, vertices are processed one by one, without knowledge of their adjacent vertices in the geometric primitive (i.e., the triangle). This stage operates by running a vertex shader (VS) on each vertex of the input stream: the VS is a custom program written in a C-like language, namely the OpenGL Shading Language (GLSL), which must be delivered to the rendering context and compiled. Here, we do not provide a complete overview of GLSL; we address the interested reader to a specialized handbook. Instead, as for the WebGL, during the rest of this book, we explain the GLSL commands and features involved in a certain piece of code, instance by instance. The following code sets up our first, simple vertex shader:
1 var vsSource = " ... "; // GLSL source code
2 var vertexShader = gl.createShader(gl.VERTEX_SHADER);
3 gl.shaderSource(vertexShader, vsSource);
4 gl.compileShader(vertexShader);
As you can notice, once a WebGLShader object is created (line 2), its source code is simply set by passing a native JavaScript string to the method shader-Source() (line 3). Finally, the shader must be compiled (line 4). The GLSL source code vsSource, for this basic example, is:
1 attribute vec2 aPosition;
2
3 void main(void)
4 {
5 gl_Position = vec4(aPosition, 0.0, 1.0);
6}
At line 1 we declare a vertex attribute named aPosition whose type is vec2, that is, a two-dimensional vector of floats. As in a C program, the main() function is the shader entry point. Every vertex shader is mandated to write to the global output variable gl_Position, a four-dimensional vector of floats (vec4) representing the vertex position and whose coordinates range from −1 to +1. In our first example, we use the two-dimensional positions (x and y) specified at step 2 and use zero and one (more on this later) for the third (z) and fourth (w) coordinates. At line 5 a C++-like constructor is used to create a vec4 from a vec2 and two scalar floats.
The vertex shader processes every vertex in the input stream, calculates its position and sends the output to the Primitive Assembler stage, whose purpose is to assemble vertices to form a geometric primitive. The Primitive Assembler then sends the primitive to the rasterizer, which interpolates the vertex shader output attributes (if any) and generates the fragments of the primitive covers on the screen.
Similarly to vertex processing, we will have to set up a fragment shader (FS) to process each generated fragment. The fragment shader setup is analogous to the vertex shader setup:
1 var fsSource = " ... "; // GLSL source code
2 var fragmentShader = gl.createShader(gl.FRAGMENT_SHADER);
3 gl.shaderSource(fragmentShader, fsSource);
4 gl.compileShader(fragmentShader);
The only change is at line 2 where we pass the FRAGMENT_SHADER symbolic constant to the creation method instead of VERTEX_SHADER.
In our first example the fragment shader simply sets the color of each fragment to blue, as shown in the following GLSL code:
1 void main(void)
2 {
3 gl_FragColor = vec4(0.0, 1.0, 0.0, 1.0);
4}
The built-in vec4 output variable gl_FragColor holds the color of the fragment. The vector component represents the red, green, blue and alpha values (RGBA) of the output color, respectively; each component is expressed as a floating point in the range [0.0,1.0].
As illustrated in Figure 2.3, vertex and fragment shaders must be encapsulated and linked into a program, represented by a WebGLProgram object:
1 var program = gl.createProgram();
2 gl.attachShader(program, vertexShader);
3 gl.attachShader(program , fragmentShader);
4 gl.bindAttribLocation(program, positionAttribIndex,"aPosition");
5 gl.linkProgram(program);
6 gl.useProgram(program);
The WebGLProgram object is created at line 1, and vertex and fragment shaders are attached (lines 2 and 3). A connection between the attribute stream slot and the vertex shader attribute is made at line 4: the bindAttribLocation() method configures the program such that the value assigned to the vertex shader attribute aPosition must be fetched from the attribute slot at index positionAttribIndex, that is, the same slot used when we configured the vertex stream in Step 2. At line 5 the program is linked, that is, the connections between the two shaders are established, and then is made as the current program at line 6.
The code in Listing 2.7 shows all the operations taken in this step.
1 function setupHowToDraw() {
2 // vertex shader
3 var vsSource = "
4 attribute vec2 aPosition;
5
6 void main(void)
7 {
8 gl_Position = vec4(aPosition , 0.0, 1.0);
9 }
10 ";
11 var vertexShader = gl.createShader(gl.VERTEX_SHADER);
12 gl.shaderSource(vertexShader, vsSource);
13 gl.compileShader(vertexShader);
14
15 // fragment shader
16 var fsSource = "
17 void main(void)
18 {
19 gl_FragColor = vec4(0.0, 1.0, 0.0, 1.0);
20 }
21 ";
22 var fragmentShader = gl.createShader(gl.FRAGMENT_SHADER);
23 gl.shaderSource(fragmentShader, fsSource);
24 gl.compileShader(fragmentShader);
25
26 // program
27 var program = gl.createProgram();
28 gl.attachShader(program, vertexShader);
29 gl.attachShader(program, fragmentShader);
30 gl.bindAttribLocation(program, positionAttribIndex,"aPosition");
31 gl.linkProgram(program);
32 gl.useProgram(program);
33}
LISTING 2.7: Complete code to program the vertex and the fragment shader.
Having configured the vertex streams and the program that will process vertices and fragments, the pipeline is now ready for drawing.
Step 4: Draw
We are now ready to draw our first triangle to the screen. This is done by the following code:
1 function draw() {
2 gl.clearColor(0.0, 0.0, 0.0, 1.0);
3 gl.clear(gl.COLOR_BUFFER_BIT);
4 gl.drawArrays(gl.TRIANGLES, 0, 3);
5 }
At line 2 we define the RGBA color to use when clearing the color buffer (line 3). The call to drawArrays() at line 4 performs the actual rendering, creating triangle primitives starting from vertex zero and consuming three vertices.
The resulting JavaScript code of all the parts is shown in Listing 2.8 for recap.
1 // global variables
2 var gl = null ;
3 var positionAttribIndex = 0;
4
5 function setupWebGL() {
6 var canvas = document.getElementById("OUTPUT-CANVAS");
7 gl = canvas.getContext("experimental-webgl");
8 }
9
10 function setupWhatToDraw() {
11 var positions = [
12 0.0, 0.0, // 1st vertex
13 1.0, 0.0, // 2nd vertex
14 0.0, 1.0 // 3rd vertex
15];
16
17 var typedPositions = new Float32Array(positions);
18
19 var positionsBuffer = gl.createBuffer();
20 gl.bindBuffer(gl.ARRAY_BUFFER, positionsBuffer);
21 gl.bufferData(gl.ARRAY_BUFFER, typedPositions,gl.STATIC_DRAW);
22
23 gl.enableVertexAttribArray(positionAttribIndex);
24 gl.vertexAttribPointer(positionAttribIndex ,
25 2, gl.FLOAT, false, 0, 0);
26}
27
28 function setupHowToDraw() {
29 // vertex shader
30 var vsSource = "
31 attribute vec2 aPosition;
32
33 void main(void)
34 {
35 gl_Position = vec4(aPosition, 0.0, 1.0);
36 }
37 ";
38 var vertexShader = gl.createShader(gl.VERTEX_SHADER);
39 gl.shaderSource(vertexShader, vsSource);
40 gl.compileShader(vertexShader);
41
42 // fragment shader
43 var fsSource = "
44 void main(void)
45 {
46 gl_FragColor = vec4(0.0, 0.0, 1.0, 1.0);
47 }
48 ";
49 var fragmentShader = gl.createShader(gl.FRAGMENT_SHADER);
50 gl.shaderSource(fragmentShader, fsSource);
51 gl.compileShader(fragmentShader);
52
53 // program
54 var program = gl.createProgram();
55 gl.attachShader(program, vertexShader);
56 gl.attachShader(program, fragmentShader);
57 gl.bindAttribLocation(program,
58 positionAttribIndex , "aPosition");
59 gl.linkProgram(program);
60 gl.useProgram(program);
61}
62
63 function draw() {
64 gl.clearColor(0.0, 0.0, 0.0, 1.0);
65 gl.clear(gl.COLOR_BUFFER_BIT);
66 gl.drawArrays(gl.TRIANGLES, 0, 3);
67}
68
69 function helloDraw() {
70 setupWebGL();
71 setupWhatToDraw();
72 setupHowToDraw();
73 draw() ;
74}
75
76 window.onload = helloDraw;
LISTING 2.8: The first rendering example using WebGL.
As anticipated in the introduction to this chapter, the amount of code necessary for displaying our first triangle seems insanely too much, but, as we will see in the following chapters, understanding these steps means knowing the largest part of every rendering procedure.
2.4 WebGL Supporting Libraries
As of now, we have seen how to set up and draw a simple triangle in WebGL. As you have seen, several lines of code to achieve this simple goal are required. This is the main reason that has motivated the birth of several WebGL libraries to ease the use of WebGL. One of the most famous ones is Three.js (http://threejs.org). Another interesting WebGL library is GLGE (http://www.glge.org). In this book, we will use SpiderGL (http://spidergl.org). SpiderGL, entirely written in Javascript, provides several modules to simplify the development of complex WebGL graphics applications. There is a module to load the content of 3D graphics data, a module of math utilities to handle matrices and vectors entities (Javascript does not provide such mathematical entities and the relative operations between them), a module to simplify the setup of the shaders for the successive rendering, and others. Note that you do not need to master this library to understand and use the code provided in the practical sections, since only very few parts of it are used. More precisely, we use this library to handle matrices and vectors and the operations between them, to handle geometric transformation, and to load and display 3D content. In the rest of the book, we will provide from time to time, some details about SpiderGL commands when they come into play in the code. For readers interested in more in-depth details we refer to the official SpiderGL Web site (http://www.spidergl.org).
2.5 Meet NVMC
The choice of an interactive video game is a fairly straightforward choice for experimenting with computer graphics theory and techniques, and there are several reasons for that:
Interactivity. Implementing a computer game imposes a hard constraint on the time spent for rendering the scene. The more the game is dynamic the more the refresh rate has to be high. For a car racing game, if the image is not refreshed at least 40–50 times per second, the game will be too little responsive and hard to play. Therefore we have a strong stimulus to find efficient solutions for rendering the scene as fast as we can.
Natural mapping between theory and practice. Computer graphics concepts are mostly introduced incrementally, that is, the content of a chapter is necessary to understand the next one. Luckily, these concepts can be applied to the construction of our video game right away, chapter after chapter. In other words the code samples that we will show in Chapter 3 will be the building blocks of those in Chapter 4, and so on. We will refer to these code samples in special sections titled “Upgrade Your Client.” All together these samples will bring us from an empty screen to a complete and advanced rendering.
Sense of achievement. As aforementioned, we will have the complete description of the scene to render and our goal will be to render it. In doing this there are countless choices to make and it is extremely unlikely that two independent developments would lead to the same result. This means that we are not just making exercises, we are creating something from scratch and shaping it to our taste. In short: Computer graphics is fun!
Completeness. It often happens that students of CG like one specific sub-topic over the others, and consequently choose to make a project on that sub-topic, neglecting the rest. This is a bad practice that cannot be pursued with the proposed car-racing game, because all the fundamentals of computer graphics are required and ineludible.
2.5.1 The Framework
NVMC is an online multi-player car racing game and as such its realization requires us to handle, other than rendering, networking, physical simulation and synchronization issues. However, these topics are beyond the scope of this book and here they are treated as a black box. This means that what follows in this section is already implemented and we only need very few notions on how to use it. The only missing part is the rendering of the scene and this is all we will care about. Figure 2.4 illustrates the architecture of the NVMC framework.
The NVMC server is in charge of the physical simulation of the race and each NVMC client corresponds to one player. The state of the game consists of the list of all the players and their state (position, velocity, damages and other per-player attributes) and other general data such as the time of day, the track condition (dry, wet, dusty etc.), and so on. A client may send commands to the server to control its car, for example TURN_LEFT, TURN_RIGHT, PAUSE etc., and these messages are input to the simulation. At fixed time steps, the server broadcasts to all the clients the state of the race so that they can render it.
2.5.2 The Class NVMC to Represent the World
All the interface towards the server is encapsulated in one single class called NVMC. Figure 2.5 shows how the elements of the scene, which are the track, the car, the trees, the buildings, etc., are accessed by the member functions of the class NVMC. The illustration is partial, since there are also other elements (the sunlight direction, the weather conditions, etc.) that are not shown there. For a complete list we refer the reader to Appendix A. Furthermore, this class provides the methods for controlling the car and takes care of the communication with the server. Note that the framework can also be used locally without connecting to any server on the network because the class NVMC also implements the server functionalities.
2.5.3 A Very Basic Client
In this section we will not introduce new WebGL notions—we will simply expand and reorganize the example of Section 2.3 to make it more modular and suitable for further development.
Figure 2.6 shows the very first NVMC client, where the car is represented with a triangle and all the rest is just a blue screen. The client is nothing other than the implementation of a single JavaScript object called NVMCClient.
Everything we will do in the rest of the book is rewriting the methods and/or extending the object NVMCClient to upgrade our client with new rendering features. In the following we will see the main methods of NVMCClient in this first very basic version.
Initializing. The method onInitialize is called once per page loading. Here we will place all the initialization of our graphics resources and data structures that need to be done once and for all. Its implementation in this very basic client is reported in Listing 2.9. The call NVMC.log at line 120 pushes a text message on the log window appearing below the canvas (see Figure 2.6). We will use this window to post information about the current version of the client and as feedback for debugging purposes. Lines 124 to 136 just create a mapping between the key W, A, S, D, and the action to take when one key is pressed. This mapping will involve more and more keys as we will have more input to take from the user (for example, switch the headlights on/off). Then at line 140 we call a function to initialize all the geometric objects we need in the client, which in this case simply means the example triangle shown in Listing 2.6 and finally at line 141 we call the function that creates a shader program.
119 NVMCClient.onInitialize = function () {
120 NVMC.log("SpiderGL Version : " + SGL_VERSION_STRING + "
");
121
122 var game = this . game ;
123
124 var handleKey = {};
125 handleKey["W"] = function (on) {
126 game.playerAccelerate = on;
127 };
128 handleKey["S"] = function (on) {
129 game.playerBrake = on;
130 };
131 handleKey["A"] = function (on) {
132 game.playerSteerLeft = on;
133 };
134 handleKey["D"] = function (on) {
135 game.playerSteerRight = on;
136 };
137 this.handleKey = handleKey;
138
139 this.stack = new SglMatrixStack();
140 this.initializeObjects(this.ui.gl);
141 this.uniformShader = new uniformShader(this.ui.gl);
142};
LISTING 2.9: The function onInitialize. This function is called once per page loading. (Code snippet from http://envymycarbook.com/chapter2/0/0.js.)
Initializing geometric objects. In these methods we take care of creating the geometric objects needed to represent the scene. We define a JavaScript object to represent a primitive consisting of a set of triangles as shown in Listing 2.10. So every geometric object will have a name, the array of vertices and triangleIndices and their respective cardinalities in numVertices and numTriangles. This representation of the geometry will be detailed later in Section 3.8 and in Section 3.9.
1 function Triangle() {
2 this.name = "Triangle";
3 this.vertices = new Float32Array([0,0,0,0.5,0,-1,-0.5,0,-1]);
4 this.triangleIndices = new Uint16Array([0 ,1 ,2]);
5 this.numVertices = 3;
6 this.numTriangles = 1;
7};
LISTING 2.10: The JavaScript object to represent a geometric primitive made of triangles (in this case, a single triangle). (Code snippet from http://envymycarbook.com/chapter2/0/triangle.js.)
Then, we define a function to create the WebGL buffers from these JavaScript objects, as shown in Listing 2.11.
35 NVMCClient.createObjectBuffers = function (gl, obj) {
36 obj.vertexBuffer = gl.createBuffer();
37 gl.bindBuffer(gl.ARRAY_BUFFER, obj.vertexBuffer);
38 gl.bufferData(gl.ARRAY_BUFFER, obj.vertices, gl.STATIC_DRAW);
39 gl.bindBuffer(gl.ARRAY_BUFFER, null);
40
41 obj.indexBufferTriangles = gl.createBuffer();
42 gl.bindBuffer(gl.ELEMENT_ARRAY_BUFFER, obj.indexBufferTriangles);
43 gl.bufferData(gl.ELEMENT_ARRAY_BUFFER, obj.triangleIndices, gl.STATIC_DRAW);
44 gl.bindBuffer(gl.ELEMENT_ARRAY_BUFFER, null);
45
46 // create edges
47 var edges = new Uint16Array(obj.numTriangles * 3 * 2);
48 for (var i = 0; i < obj.numTriangles; ++i) {
49 edges[i * 6 + 0] = obj.triangleIndices[i * 3 + 0];
50 edges[i * 6 + 1] = obj.triangleIndices[i * 3 + 1];
51 edges[i * 6 + 2] = obj.triangleIndices[i * 3 + 0];
52 edges[i * 6 + 3] = obj.triangleIndices[i * 3 + 2];
53 edges[i * 6 + 4] = obj.triangleIndices[i * 3 + 1];
54 edges[i * 6 + 5] = obj.triangleIndices[i * 3 + 2];
55}
56
57 obj.indexBufferEdges = gl.createBuffer();
58 gl.bindBuffer(gl.ELEMENT_ARRAY_BUFFER, obj.indexBufferEdges);
59 gl.bufferData(gl.ELEMENT_ARRAY_BUFFER, edges, gl.STATIC_DRAW);
60 gl.bindBuffer(gl.ELEMENT_ARRAY_BUFFER, null);
61};
LISTING 2.11: Creating the objects to be drawn. (Code snippet from http://envymycarbook.com/chapter2/0/0.js.)
These two functions are a more modular implementation of the function setupWhatToDraw() in Listing 2.6 to both create the JavaScript objects and their WebGL counterparts. At lines 36, 41 and 57 we extend the JavaScript object passed as input (in this case an object triangle) with the WebGL buffers for vertices, triangles and edges of the object. The edges are inferred from a list of triangles, which means that for each triangle indicated with indices (i,j,k), three edges are created: the edge (i, j) the edge (j, k) and the edge (k, i). In this implementation we do not care that if two triangles share one edge, we will have two copies of the same edge.
In Listing 2.12 we show how the functions above are used for our first client.
63 NVMCClient.initializeObjects = function (gl) {
64 this.triangle = new Triangle ();
65 this.createObjectBuffers(gl , this.triangle);
66};
LISTING 2.12: Creating geometric objects. (Code snippet from http://envymycarbook.com/chapter2/0.js.)
Rendering. In Listing 2.13 we have the function drawObject to actually perform the rendering. The difference from the example in Listing 2.7 is that we render both the triangles and their edges and we pass the color to use (as fillColor and lineColor). So far the only data we passed from our JavaScript code to the program shader were vertex attributes, more specifically the position of the vertices. This time we also want to pass the color to use. This is a global data, meaning that it is the same for all the vertices processed in the vertex shader or for all the fragments of the fragment shader. A variable of this sort must be declared by using the GLSL keyword uniform. Then, when the shader program has been linked, we can query it to know the handle of the variable with the function gl.getUniformLocation (see line 52 in Listing 2.14) and we can use this handle to set its value by using the function gl.uniform (see lines 20 and 25 in Listing 2.13). Note that the gl.uniform function name is followed by a postfix, which indicates the type of parameters the function takes. For example, 4fv means a vector of 4 floating points, 1i means an integer, and so on.
10 NVMCClient.drawObject = function (gl, obj, fillColor, lineColor) {
11 gl.bindBuffer(gl.ARRAY_BUFFER, obj.vertexBuffer);
12 gl.enableVertexAttribArray(this.uniformShader.aPositionIndex);
13 gl.vertexAttribPointer(this.uniformShader.aPositionIndex, 3, gl.FLOAT, false, 0, 0);
14
15 gl.enable(gl.POLYGON_OFFSET_FILL);
16
17 gl.polygonOffset (1.0, 1.0);
18
19 gl.bindBuffer(gl.ELEMENT_ARRAY_BUFFER , obj.indexBufferTriangles);
20 gl.uniform4fv(this.uniformShader.uColorLocation , fillColor);
21 gl.drawElements(gl.TRIANGLES , obj.triangleIndices.length , gl.UNSIGNED_SHORT , 0);
22
23 gl.disable(gl.POLYGON_OFFSET_FILL);
24
25 gl.uniform4fv(this.uniformShader.uColorLocation, lineColor);
26 gl.bindBuffer(gl.ELEMENT_ARRAY_BUFFER, obj.indexBufferEdges);
27 gl.drawElements(gl.LINES, obj.numTriangles * 3 * 2, gl.UNSIGNED_SHORT , 0) ;
28
29 gl.bindBuffer(gl.ELEMENT_ARRAY_BUFFER, null);
30
31 gl.disableVertexAttribArray(this.uniformShader.aPositionIndex) ;
32 gl.bind Buffer(gl.ARRAY BUFFER, null);
33};
LISTING 2.13: Rendering of one geometric object. (Code snippet from http://envymycarbook.com/chapter2/0.js.)
In the NVMC clients the shader programs will be encapsulated in JavaScript objects, as you can see in Listing 2.14, so that we can exploit a common interface to access the members (for example, the position of the vertices will always be called aPositionIndex on every shader we will write).
1 uniformShader = function (gl) {
2 var vertexShaderSource = "
3 uniform mat4 uModelViewMatrix;
4 uniform mat4 uProjectionMatrix;
5 attribute vec3 aPosition;
6 void main(void)
7 {
8 gl_Position = uProjectionMatrix *
9 uModelViewMatrix * vec4(aPosition, 1.0);
10 }
11 ";
12
13 var fragmentShaderSource = "
14 precision highp float;
15 uniform vec4 uColor;
16 void main(void)
17 {
18 gl_FragColor = vec4(uColor);
19 }
20 ";
21
22 // create the vertex shader
23 var vertexShader = gl.createShader(gl.VERTEX_SHADER);
24 gl.shaderSource(vertexShader, vertexShaderSource);
25 gl.compileShader(vertexShader);
26
27 // create the fragment shader
28 var fragmentShader = gl.createShader(gl.FRAGMENT_SHADER);
29 gl.shaderSource(fragmentShader, fragmentShaderSource);
30 gl.compileShader(fragmentShader);
31
32 // Create the shader program
33 var aPositionIndex = 0;
34 vai shaderProgram = gl.createProgram();
35 gl.attachShader(shaderProgram, vertexShader);
36 gl.attachShader(shaderProgram, fragmentShader);
37 gl.bindAttribLocation(shaderProgram, aPositionIndex, "aPosition");
38 gl.linkProgram(shaderProgram);
39
40 // If creating the shader program failed, alert
41 if (!gl.getProgramParameter(shaderProgram, gl.LINK_STATUS)) {
42 var str = "Unable to initialize the shader program.
";
43 str += "VS:
" + gl.getShaderInfoLog(vertexShader) + "
";
44 str += "FS:
" + gl.getShaderInfoLog(fragmentShader) + "
";
45 str += "PROG:
" + gl.getProgramInfoLog(shaderProgram);
46 alert(str) ;
47}
48
49 shaderProgram.aPositionIndex = aPositionIndex;
50 shaderProgram.uModelViewMatrixLocation = gl.getUniformLocation (shaderProgram , "uModelViewMatrix");
51 shaderProgram.uProjectionMatrixLocation = gl.getUniformLocation(shaderProgram , "uProjectionMatrix");
52 shaderProgram.uColorLocation = gl.getUniformLocation shaderProgram, "uColor");
53
54 return shaderProgram;
55};
LISTING 2.14: Program shader for rendering. (Code snippet from http://envymycarbook.com/chapter2/0/0.js.)
Interface with the game. The class NVMCClient has a member game that refers to the class NVMC and hence gives us access to the world both for giving input to the simulation and for reading information about the scene. In this particular client we only read the position of the player’s car in the following line of code (Listing 2.15):
94 var pos = this.myPos()
LISTING 2.15: Accessing the elements of the scene. (Code snippet from http://envymycarbook.com/chapter2/0/0.js.)
2.5.4 Code Organization
The various NVMC clients are organized in the following way: each client corresponds to a folder. The clients are separated for each chapter and are numbered starting from 0 in each chapter (see Figure 2.7). So, for example, the second client (that is, the client number is 1) of chapter X corresponds to the folder chapterX/1. Inside the folder of each client we have the HTML file [client numberj.html, a file named shaders.js for the shaders introduced with the client, one or more javascript files containing the code for new geometric primitives introduced within the client and a file [client number].js containing the implementation for the class NVMCClient.
Note, and this is very important, that each file [client number].js contains only the modifications with respect to the previous client while in the HTML file we explicitly include the previous versions of the class NVMCClient, so that everything previously defined in each client file will be parsed. This is very handy because it allows us to write only the new parts that enrich our client and/or to redefine existing functions. For example, the function createObjectBuffers will not need to be changed until chapter 5 and so it will not appear in the code of the clients of chapter 4. A reader may argue that many functions can be parsed without actually being called because they are overwritten by more recent versions. Even if this is a useless processing that slows down the loading time of the Web page, we prefer to proceed in this way for didactic purposes. Nothing prevents you from removing overwritten members when a version of the client is finalized.
On the contrary, the shader programs are not written incrementally, since we do not want to always use an improved version of the same shader program but, instead, we will often use many of them in the same client. The same goes for the geometric objects. In this first client we introduced the Triangle, in the next chapter we will write the Cube, the Cone and other simple primitives and we will use them in our future clients.