Skeletal Animation
Watching a model spin is fun. Even better is watching it walk, jump, and scratch its head. In such skeletal animations, the model bends and twists as if it had real bones and muscles at work under its skin. Skeletal animations are crafted by a 3D artist and saved in the model file rather than programmed. In an earlier chapter, we used our glTF loader to read in static models. Now we use the same loader to read in models with skeletal animations.
Animators don't move individual vertices when composing a skeletal animation. Rather, they insert a virtual skeleton inside the model. Each vertex on the skin is tied to one or more of the skeleton's bones. When the artist bends or rotates a bone at a joint, the vertices tied to it follow.
To keep animation tractable, animation systems generally restrict each vertex to being influenced by no more than four bones. Each bone has a transformation matrix that changes during the animation's playback. In the vertex shader, we transform each vertex by its four bones' matrices. This moves the model from model space into pose space.
Let's update our code to play back a skeletal animation. The glTF loader gives a mesh two vertex attributes that we need: joints and weights. The joints attribute is 4-vector of bone indices. The weights attribute is a 4-vector that quantifies its bones' influences. If a vertex is tied to only a single bone, it will still have four bone indices, but three of the weights will be 0. We add these attributes with code like this:
import * as gltf from './gltf.js';
let model: gltf.Model;
async function initializeModel(url: string) {
model = await gltf.Model.readFromUrl(url);
const attributes = new VertexAttributes();
attributes.addAttribute('position', model.meshes[0].positions.count, 3, model.meshes[0].positions.buffer);
attributes.addAttribute('normal', model.meshes[0].normals!.count, 3, model.meshes[0].normals!.buffer);
attributes.addAttribute('weights', model.meshes[0].weights!.count, 4, model.meshes[0].weights!.buffer);
attributes.addAttribute('joints', model.meshes[0].joints!.count, 4, new Float32Array(model.meshes[0].joints!.buffer));
attributes.addIndices(model.meshes[0].indices!.buffer);
vao = new VertexArray(shader, attributes);
}import * as gltf from './gltf.js';
let model: gltf.Model;
async function initializeModel(url: string) {
model = await gltf.Model.readFromUrl(url);
const attributes = new VertexAttributes();
attributes.addAttribute('position', model.meshes[0].positions.count, 3, model.meshes[0].positions.buffer);
attributes.addAttribute('normal', model.meshes[0].normals!.count, 3, model.meshes[0].normals!.buffer);
attributes.addAttribute('weights', model.meshes[0].weights!.count, 4, model.meshes[0].weights!.buffer);
attributes.addAttribute('joints', model.meshes[0].joints!.count, 4, new Float32Array(model.meshes[0].joints!.buffer));
attributes.addIndices(model.meshes[0].indices!.buffer);
vao = new VertexArray(shader, attributes);
}The model will have some number of animation clips. We can see a list of their names by printing their keys:
for (let clip of Object.keys(model.animations)) {
console.log(clip);
}for (let clip of Object.keys(model.animations)) {
console.log(clip);
}Suppose our model has a clip named jump. We set it in motion by calling the play method:
model.play('jump');model.play('jump');This makes the animation active in the data structure, but it doesn't update any animation state. To do that, we must call the tick method on each frame of our continuous rendering:
function animate(now: DOMHighResTimeStamp) {
const elapsed = then ? now - then : 0;
model.tick(elapsed);
render();
requestAnimationFrame(animate);
then = now;
}function animate(now: DOMHighResTimeStamp) {
const elapsed = then ? now - then : 0;
model.tick(elapsed);
render();
requestAnimationFrame(animate);
then = now;
}In the render function, right before we draw the the model, we upload all the bone matrices as uniforms at the same time we upload our other matrices:
// Upload three standard matrices
shader.setUniformMatrix4fv('clipFromEye', clipFromEye.buffer());
shader.setUniformMatrix4fv('eyeFromWorld', eyeFromWorld.buffer());
shader.setUniformMatrix4fv('worldFromPose', worldFromPose.buffer());
// Upload bone matrices
for (let [i, matrix] of model.skinTransforms(300).entries()) {
shader.setUniformMatrix4fv(`jointTransforms[${i}]`, matrix.buffer());
}// Upload three standard matrices
shader.setUniformMatrix4fv('clipFromEye', clipFromEye.buffer());
shader.setUniformMatrix4fv('eyeFromWorld', eyeFromWorld.buffer());
shader.setUniformMatrix4fv('worldFromPose', worldFromPose.buffer());
// Upload bone matrices
for (let [i, matrix] of model.skinTransforms(300).entries()) {
shader.setUniformMatrix4fv(`jointTransforms[${i}]`, matrix.buffer());
}Notice three things about this code. First, there are brackets and an index in the uniform name. The uniform is actually an array of matrices, as we're about to see. We upload them one at a time. Second, skinTransforms method expects a parameter describing how much time we allow two clips to overlap. An abrupt transition from running to idling, for example, would be jarring. Third, the worldFromModel matrix has been renamed worldFromPose. The matrix gets applied not to the raw model but to the posed model.
In the vertex shader, we make several changes: we declare a jointTransforms array as a uniform, we declare joints and weights vertex attributes as in variables of type vec4, and we multiply the bone matrices together to form the poseFromModel matrix that we add to our transformation chain:
uniform mat4 clipFromEye;
uniform mat4 eyeFromWorld;
uniform mat4 worldFromPose;
uniform mat4 jointTransforms[JOINT_TRANSFORM_COUNT];
in vec3 position;
in vec3 normal;
in vec4 joints;
in vec4 weights;
out vec3 mixNormal;
void main() {
mat4 poseFromModel =
weights.x * jointTransforms[int(joints.x)] +
weights.y * jointTransforms[int(joints.y)] +
weights.z * jointTransforms[int(joints.z)] +
weights.w * jointTransforms[int(joints.w)];
gl_Position = clipFromEye * eyeFromWorld * worldFromPose * poseFromModel * vec4(position, 1.0);
mixNormal = normal;
}uniform mat4 clipFromEye;
uniform mat4 eyeFromWorld;
uniform mat4 worldFromPose;
uniform mat4 jointTransforms[JOINT_TRANSFORM_COUNT];
in vec3 position;
in vec3 normal;
in vec4 joints;
in vec4 weights;
out vec3 mixNormal;
void main() {
mat4 poseFromModel =
weights.x * jointTransforms[int(joints.x)] +
weights.y * jointTransforms[int(joints.y)] +
weights.z * jointTransforms[int(joints.z)] +
weights.w * jointTransforms[int(joints.w)];
gl_Position = clipFromEye * eyeFromWorld * worldFromPose * poseFromModel * vec4(position, 1.0);
mixNormal = normal;
}The glTF loader does most of the heavy lifting for us, but it does call upon our matrix and quaternion code to build the transformations.