Magnifying Textures

Suppose some friends send us a photo of their new baby. Suppose further that we always complain to them about the size of photo attachments. Accordingly, they scale the photo of their baby down to just 2×2. When we display it fullscreen on our phone, what will it look like? Probably something like this:

What a beautiful baby blur. There's a severe mismatch between the amount of information in the photo and the size of the display showing the image. The image only offers four colors, whereas the display above must find enough colors to fill a square that is 300×300.

There are several strategies for coming up with new colors. All strategies use information from the surrounding neighborhood of known pixels. The process of picking new colors when resizing an image is called filtering. When the resized image is bigger than the original, we apply a magnification filter. One filtering strategy that we've already seen is bilinear interpolation, which performs a weighted blend of the four surrounding known pixels. Blerped by definition smooths out an image. Taken to an extreme, the image is more blur than baby.

Another strategy is to pick the known pixel from the image that is nearest to the pixel in the resized image. This is called nearest neighbor interpolation. Nearest neighbor interpolation gives images a pixelated appearance:

Gamers feel a lot of nostalgia for games that were published before graphics cards came along, when pixel art was the de facto art style and 3D models were too expensive to render at interactive frame rates. Nearest neighbor interpolation is still heavily used in 2D games.

Textures behave just like baby photos. When a texture is pasted onto a very large surface that has more fragments than the texture has texels, the texturing hardware has to magnify the texture. Some fragments will fall between texels, and the graphics card will have to make up the in-between colors. We tell the texture unit to use bilinear interpolation to make up the color with this statement:

gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.LINEAR);

gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.LINEAR);

If the texture is meant to appear pixelated, we use gl.NEAREST instead of gl.LINEAR:

gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.NEAREST);

gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.NEAREST);

Examine how the two filter parameters influence the appearance of this RPG character:

Nothing in our shader changes. For both interpolation schemes, we still call the texture function on a sampler2D.

Each filtering strategy has a weakness. Linear filtering is blurry under extreme magnification. Nearest filtering produces jagged lines when texels don't align with the framebuffer grid. Rotate the character and you will see these jagged lines dance around. Neither can help us see a baby in a 2×2 photo.