Raycasting

Sometimes we need to know precisely what location the mouse hits in the scene, perhaps to fire a projectile or place a vertex. However, the mouse does not correspond to just a single location in the scene. An infinite number of locations project onto that one pixel. These locations are all on a single ray or line segment that starts where the mouse lands on the near clipping and passes through the viewing frustum until exiting out the far clipping plane.

To figure out which locations the mouse is over, we walk along this ray and see what it intersects. This technique is called raycasting. Explore the idea of casting a ray through the mouse into the scene by clicking on this renderer:

On each click, a cylinder is added to the scene. It bridges the mouse's entrance and exit points in the viewing frustum. Rotate the scene to see the complete cylinders.

The mouse listener in this renderer walks the mouse position backward through the transformation pipeline by inverting the transformations. It first turns the mouse's pixel space coordinates into normalized device coordinates:

function onMouseUp(event: MouseEvent) {
  const mousePixel = [
    event.clientX,
    canvas.height - event.clientY
  ];

  const mouseNormalized = new Vector4(
    mousePixel[0] / canvas.width * 2 - 1,
    mousePixel[1] / canvas.height * 2 - 1,
    -1,
    1,
  );

  // ...
}

function onMouseUp(event: MouseEvent) {
  const mousePixel = [
    event.clientX,
    canvas.height - event.clientY
  ];

  const mouseNormalized = new Vector4(
    mousePixel[0] / canvas.width * 2 - 1,
    mousePixel[1] / canvas.height * 2 - 1,
    -1,
    1,
  );

  // ...
}

The mouse operates in two dimensions and therefore doesn't have a z-coordinate of its own. This listener hardcodes its normalized z-coordinate to -1, which puts it on the near clipping plane on the unit box. The homogeneous coordinate is set to 1 in order to treat the mouse coordinates as a position rather than a vector.

Then the normalized position is untransformed by the inverses of the matrices:

function onMouseUp(event: MouseEvent) {
  // ...

  let mouseEye = eyeFromClip.multiplyVector(mouseNormalized);
  mouseEye = mouseEye.divideScalar(mouseEye.w);
  const mouseWorld = worldFromEye.multiplyVector(mouseEye);

  let rayStart = mouseWorld;
}

function onMouseUp(event: MouseEvent) {
  // ...

  let mouseEye = eyeFromClip.multiplyVector(mouseNormalized);
  mouseEye = mouseEye.divideScalar(mouseEye.w);
  const mouseWorld = worldFromEye.multiplyVector(mouseEye);

  let rayStart = mouseWorld;
}

Remember how we skipped over clip space and the perspective divide earlier? To correct for that skipping, we must divide the eye space position by its homogeneous coordinate.

The ray starts at rayStart. It heads to the rayEnd position on the far clipping plane that projects to the mouse position. We find this position just as we did rayStart. The only difference is that the z-coordinate of the normalized position is 1:

function onMouseUp(event) {
  // ...

  mouseNormalized.z = 1;
  mouseEye = eyeFromClip.multiplyVector(mouseNormalized);
  mouseEye = mouseEye.divideScalar(mouseEye.w);
  mouseWorld = worldFromEye.multiplyVector(mouseEye);
  let rayEnd = mouseWorld;

  // add in a cylinder spanning rayStart to rayEnd
}

function onMouseUp(event) {
  // ...

  mouseNormalized.z = 1;
  mouseEye = eyeFromClip.multiplyVector(mouseNormalized);
  mouseEye = mouseEye.divideScalar(mouseEye.w);
  mouseWorld = worldFromEye.multiplyVector(mouseEye);
  let rayEnd = mouseWorld;

  // add in a cylinder spanning rayStart to rayEnd
}

The renderer above fits a cylinder between the rayStart and rayEnd positions. More sophisticated renders will construct a ray and perform collision detection or launch a projectile along the ray.