Magnifying Textures
Some friends send a photo of their new baby. Since we are always complaining about the size of photo attachments, they scaled the image down to just 2 pixels wide and 2 pixels tall. When we display the photo fullscreen on our phone, what will it look like? Probably something like this:
What a beautiful baby blur. There's a severe mismatch between the amount of information in the photo and the size of the display showing the image. The image only offers four colors, whereas the display above must find enough colors to fill a square that is 300 by 300 pixels.
There are several strategies for coming up with new colors. All strategies use information from the surrounding neighborhood of known pixels. The process of picking a color based on the surrounding neighborhood is called filtering or interpolation.
One filtering strategy is to perform a weighted blend of the known pixels. This blending is called linear interpolation. The closer one is to one of the known pixels, the stronger its contribution to the interpolated color. Linearly interpolated images are blurry when there aren't enough pixels.
Another strategy is to show the known pixel from the photo that is nearest to the pixel in the display. This is called nearest neighbor interpolation. Nearest neighbor interpolation gives images a pixelated appearance:
Gamers feel a lot of nostalgia for games that were published before graphics cards came along, when pixel art ruled the Earth and 3D models were too expensive to render at interactive framerates. Nearest neighbor interpolation is still heavily used in 2D games.
Textures behave just like the baby photo. When a texture is pasted onto a very large surface that has more pixels than the texture has texels, the texturing hardware has to magnify the texture. Some pixels will fall between texels, and the graphics card will have to make up the in-between colors. We tell the texture unit to use linear interpolation to make up the color with this statement:
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.LINEAR);gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.LINEAR);
If the texture is meant to appear pixelated, we use gl.NEAREST instead of gl.LINEAR:
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.NEAREST);gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.NEAREST);
Examine how the two filter parameters influence the appearance of this RPG character:
Nothing in our shader changes. For both interpolation schemes, we still call the texture function on a sampler2D.
Each filter has a weakness. Linear filtering looks out of focus under extreme magnification. Nearest filtering produces jagged lines when texels don't align with the framebuffer grid. Rotate the character and you will see these jagged lines dance around. We may be able to strike a happy medium by scaling up the pixel art so each pixel doubles or quadruples in size. Then we enable linear filtering. The sharp transitions between pixels are smooth, but those transitions shrink in size and therefore appear less blurry.