Shader is a small program that allows you to unleash the computation power of GPUs. This power exceeds the computation capabilities of the most powerful CPUs. So using shaders is crucial for creating competitive and astonishing projects. This chapter will cover the basics on using shaders in openFrameworks for creating 2D video effects and 3D object deformations. We will cover the following topics:
- Structure of a shader's code
- An example of a simple fragment shader
- Creating video effects with fragment shaders
- Deforming objects with a vertex shader
- Using a geometry shader
Shaders are small programs executed on a Graphics Processing Unit (GPU), which is located on the video card. Shaders work when OpenGL renders something on the screen (or in the screen buffer), and they modify the geometry and pixels of the rendered objects. They work very fast and perform advanced processing of images and complex 3D scenes at faster rates, which is impossible using today's CPUs. This is the reason that shaders are widely used for interactive rendering and VJing.
Shaders are written using Graphics Library Shading Language (GLSL), which is actually a C++ language, and are extended with vector and matrix types and mathematical operations. Some of the C++ features, such as working with memory (pointers, references, and the new operator) and classes, are not included in GLSL.
When shaders are used in openFrameworks, the shaders' codes are stored in the text files in the data folder of the project. Unlike an ordinary C++ program, shaders' code is compiled at runtime, just when it's needed. So you can change the shaders' code and restart your project without recompiling the openFrameworks' project.
No special GLSL compiler is needed because it is embedded in all the modern video card drivers. So shaders are universal; once written, they can be used in many interactive software platforms such as openFrameworks, Processing, Cinder, Quartz Composer, Touch Designer, and vvvv.
Tip
You can run and edit simple shaders right in your browser using online shader sandboxes such as http://glsl.heroku.com. They contain galleries with examples of great shaders which you can learn and use in your projects. For example, try http://glsl.heroku.com/e#8801.0.
Working with shaders in openFrameworks is simple with the class ofShader. It lets you load and compile the shaders, enable the shaders and set their parameters, and finally disable the shaders.
There are several types of shaders. Each of them works in a particular part of the rendering pipeline. You can use only one shader of a given type at a particular moment. Nevertheless, you can use many different shaders of the same type by switching between them.
In this chapter, we will consider only three types of shaders, which are the most popular and are used in most applications:
- A vertex shader processes each vertex of the rendered object and changes its properties, such as position, normal, color, and some custom attributes. It can be used for geometric transformations of 3D objects. See the Deforming objects with a vertex shader section for details.
- A geometry shader gets the list of vertices of a primitive to be rendered (for example, the three vertices of a triangle), and generates a new list of vertices, forming one or several primitives that will actually be rendered instead of the one that was input. For example, it can replace each rendered triangle with a bunch of lines that form a "furry" surface. See the Using a geometry shader section for details.
- A fragment shader processes the color and depth of a pixel which is ready to be rendered to the screen or screen buffer. A fragment shader can be used for implementing postprocessing effects, and also for more complex image processing and generation. See the Creating video effects with fragment shaders section for details.
Tip
See the full list of the different types of shaders in Chapter 2, Overview of OpenGL Shading of the OpenGL Shading Language Specification document. Currently, the latest specification can be downloaded at http://www.opengl.org/registry/doc/GLSLangSpec.4.30.6.pdf. Check newer versions and other documents at http://www.opengl.org/documentation/glsl.
Though all the shaders are written in one GLSL language, different types of shaders have different built-in input and output variables and special commands in their syntax.
Note
Shaders work in a particular order, that is, first the vertex shader, then the geometry shader, and finally the fragment shader. If you want to use the vertex or fragment shader, you need to specify and enable both of them. If you want to use the geometry shader, you need to specify and enable all three types of shaders. This is not a problem because when you are interested in one particular type of shader, you can use "dummy" shaders for the other shader types.
When deciding whether you should use the shaders technology in your openFrameworks project or not, take into account the following considerations:
- If you need to make a vibrant real-time visualization, including the transforming and pulsating of 3D objects or images, then using shaders for the effects is most probably the best choice.
- If you have some shader code and want to try it in your project, you can often embed the shader without it being changed but sometimes, a little modification is needed in the shader's code.
- If you have a working project and are manipulating the project with 2D graphics or 3D objects' geometry using CPU, and you notice that it works too slowly, then you can move a part of the graphics computations to shaders. Then the CPU usage will decrease, and the overall application performance will (often radically) improve.
- If you need to perform massive nongraphics computations, such as simulating the physics of the million particles, you still can do it with shaders. For such a purpose, you need a way for retaining shaders' processing results:
- To retain data from the vertex and geometry shaders, use OpenGL's Transform Feedback feature (see details in OpenGL Wiki at http://www.opengl.org/wiki/Transform_Feedback)
- To retain data from a fragment shader, perform rendering in an offscreen buffer (see details in the Using FBO for offscreen drawing section in Chapter 2, Drawing in 2D)
Alternatively, instead of vertex, geometry or fragment shaders, you can use compute shaders, which let you perform universal computations and output their results in your custom arrays.
Anyway, shaders and other GPU-programming technologies are the dominant topics in modern computing and supercomputing in the near future. So we highly recommend learning them.
Now let's consider the structure of the simplest fragment and vertex shaders and some basics of GLSL language. (A geometry shader has a similar structure, and so is not discussed in this section. See the Using a geometry shader section for details.)