Verbs and Nouns

Earlier we defined computer graphics as a discipline concerned with turning geometric models into images with the help of a computer. Let's break down this transformation into four discrete stages: model, assemble, rasterize, and display. As we discuss each of these verbs, we'll also encounter some nouns that graphics developers use to identify picture-making hardware and important data.

Model

The shape to be displayed is first described in a model. In an industrial setting, a model is usually designed by a 3D artist who views the shape as a sculpture to be worked by a mouse or stylus. The model is saved to a file and handed off to a software developer, who views the shape as a collection of numbers.

The numbers represent spatial or visual properties measured at the shape's vertices. A vertex is a location on the shape's surface that is significant in some way, usually because its where the surface bends or its color changes. With more vertices, an artist can show more detail. With fewer vertices, images will render more quickly.

An artist must strike a balance between performance and realism by removing vertices that don't contribute any new information. Squares, for example, are usually modeled with vertices only at the corners. Why put a vertex in the middle if the properties there can be calculated by interpolating between or blending the properties of the corners?

Typical properties include a vertex's position in Cartesian coordinates and color. Here's a model of a single triangle that occupies the \(z = 0\) plane, whose bottom-left vertex is at the origin, and whose vertices are colored red, green, and blue:

The software developer writes code to read these numbers from disk and to upload them into vertex buffers. A vertex buffer is a flat array of bytes stored in the memory of the graphics card (video RAM, or VRAM). As much as possible, graphics developers want data to reside on the graphics card itself, because pulling numbers from VRAM is a lot faster than pulling them from RAM.

Assemble

With the vertex data situated on the graphics card, the developer issues a draw routine that pulls vertex properties out of the flat array and assembles them into geometric primitives. The most common primitive is a triangle, but WebGL also supports drawing points and lines. We focus on points and lines in this chapter and triangles in the next.

By itself, a vertex buffer gives no indication of how its vertices are supposed to connect together. The simplest option is to let the graphics card connect them together in sequence according to the type of geometric primitive being rendered. WebGL lets you choose between four different point and line primitives:

• POINTS → each vertex is processed in isolation
• LINES → vertices are processed in pairs to form unconnected line segments
• LINE_STRIP → vertices are processed as a polyline, a connected chain of line segments
• LINE_LOOP → vertices are processed as a closed polyline, with the last vertex connecting back to the first

Consider this vertex buffer, which holds the positions and colors of five vertices:

As a sequence of POINTS, this data contains five separate vertices. As a sequence of LINES, it contains two line segments, one between vertices 0 and 1, and another between vertices 2 and 3. Vertex 4 doesn't have a mate, so it is ignored. As a LINE_STRIP, it contains a four-segment chain. As a LINE_LOOP, it contains a pentagon.

Explore how the vertex buffer is interpreted according to the primitive type in this renderer:

Rasterize

Elsewhere in VRAM is the framebuffer, which is the rectangular grid of pixels that will be displayed to the user. Rectangular grids are sometimes called rasters. Drawing a geometric primitive into the framebuffer is rasterization.

Once the card has determined which vertices belong to a primitive, it runs each vertex through a vertex shader. In the early days of graphics libraries, the vertex shader was a fixed operation in the graphics card's driver software, and developers had limited influence over its behavior. In the mid-2000s, the graphics pipeline became programmable. Developers started writing custom vertex shaders to add new visual effects.

The primary purpose of the vertex shader is to turn the properties stored in the vertex buffer into position and lighting data for a single vertex. The card runs the vertex shader on all the vertices of a primitive, examines where the vertices land on the framebuffer, and iterates through the pixels between these locations. Each covered pixel is run through the fragment shader. The fragment shader takes as input the interpolated outputs from the vertex shader and outputs the pixel's color according to its material properties and relationship to light sources.

Graphics programs tend to be computationally intense. They have to run each vertex through the vertex shader and each pixel through the fragment shader. That's a lot of data to process, and the framebuffer may need to be updated 60 times a second to feel interactive. To offload this computation from the main CPU, companies began selling rasterization hardware to consumers in the 1990s. These graphics cards contain a special chip called a graphics processing unit, or GPU. The instruction set of a GPU is much narrower than that of a CPU. It is designed to multiply matrices, interpolate values between vertices, and read and write from image buffers. GPUs have many vertex and fragment processors that run in parallel.

Display

The word pixel is a contraction of picture element. We normally think of a pixel as a colored square, but this isn't technically accurate. A pixel is really just an infinitely small observation of color in some two-dimensional space. For this sample to be visible to a human, it must be spread across a light-emitting display. The graphics card and display hardware take care of this for us, and the process is often more complex than subdividing the screen into a neat rectangle of independent squares. We will not explore the complexities of displaying pixels in this course. Instead, we'll spend most our time with the first three verbs and just let WebGL magically light up the display with the pixel colors that we write to the framebuffer.

We've walked through the broad stages of the rendering process. Let's make them more concrete by rendering some geometry. Our first step will be to create some utility classes for vertex properties and shader programs.