Lecture: Motion Capture

Dear students:

Last week we visited the Wearables Computing Lab, and several of us recorded ourselves with a motion capture system. The output was a BVH file. Today we'll explore how to process these files and produce a glTF file that we can render in the same way that we've rendered other models with skeletal animations.

Processing BVH

Converting a BVH file to a glTF file is involved enough that we should explore it together. It's also tedious, so merely watching me do it would be a snoozefest. Therefore, our format today will be a little different. We'll do a guided walkthough. I'll demonstrate each step, and then you'll play along on your own computer.

Here are the steps:

1. Download the starting Blender file and BVH file of your classmate.
2. Open the Blender file and import the BVH file.
3. We need to add a character model to fit the armature, but first we need he armature to be in a rest pose with no transformations applied.
   1. Scrub the animation timeline to frame 0.
   2. Enter Pose Mode.
   3. Select Pose / Clear Transforms / All.
   4. Hit I to record a keyframe.
4. Return to object mode, select the default cube, and enter edit mode. Turn it into a character model by extruding and manipulating. Fit the limbs so the geometric joints occur at the bone joints. That way we can clearly identify the vertices that a bone must influence. Use View / Viewpoint to get axis-aligned orthographic views that let us operate in just two dimensions at a time.
5. The character and armature are currently independent, but the character must be a child of the armature. In object mode, select the character first and shift-select the armature second. Select Object / Parent / Armature Deform / With Automatic Weights.
6. The automatic weight calculation doesn't always do the right thing, and we may need to adjust the weights.
   1. Play the very big single animation that came in with the BVH file.
   2. Does the model bend appropriately? If not, select the model and enter weight paint mode.
   3. We must inspect each bone and see its influence, but selecting the bones can be tricky. In the Properties view, select the Data panel. Click on a vertex group corresponding to a bone.
   4. Paint the vertices around each joint with a weight of 1 and all other vertices with a weight of 0.
7. The animation from the capture is one big action. We need to break it up into separate clips.
   1. Switch to the Animation layout using the menu at the top.
   2. Switch from the Dope Sheet to the Action Editor.
   3. Identify the starting keyframe of an animation.
   4. Click the New Action button and give it a meaningful name.
   5. Scrub to the preceding keyframe, choose Select / Before Current Frame, and delete all keyframes before the clip.
   6. Scrub to the frame just after the last, choose Select / After Current Frame, and delete all keyframes after the clip.
   7. Select all remaining keyframes and move them to frame 1.
   8. Repeat this process for all clips.
8. Motion capture animations are unlikely to loop perfectly. As much as possible, we want a model to start and end in the same pose. If we can do this with our bodies at the time of capture, great. But we will fail. An alternative is to blend the final keyframes back to the starting keyframe.
   1. Select the starting keyframe and copy it.
   2. Scrub to just after the ending keyframe and paste.
   3. Select some number of final keyframes that to linearly ramp back to the start.
   4. Paste this script into the Python console:
      content_copyPython

      import bpy
      
      armature = bpy.context.active_object
      fcurves = armature.animation_data.action.fcurves
      
      for curve in fcurves:
          keys = curve.keyframe_points
          selectedKeys = []
          for k in keys:
              if k.select_control_point:
                 selectedKeys.append(k)
          lastVal = selectedKeys[-1].co.y
          step = 1 / len(selectedKeys)
          influence = 0
          for key in selectedKeys:
              difference = lastVal - key.co.y
              key.co.y = key.co.y + (difference * influence)
              influence += step

      import bpy
      
      armature = bpy.context.active_object
      fcurves = armature.animation_data.action.fcurves
      
      for curve in fcurves:
          keys = curve.keyframe_points
          selectedKeys = []
          for k in keys:
              if k.select_control_point:
                 selectedKeys.append(k)
          lastVal = selectedKeys[-1].co.y
          step = 1 / len(selectedKeys)
          influence = 0
          for key in selectedKeys:
              difference = lastVal - key.co.y
              key.co.y = key.co.y + (difference * influence)
              influence += step

      Credit for the script goes to user Latin1 on the Blender Artists forum.
9. Export a glTF file. But we don't want to include the big clip. It will slow things down and consume disk storage. In the export dialog, under Animation / Action Filter, select just the smaller clips.

TODO

Here's your list of things to do before we meet next:

See you next time.

P.S. It's time for a haiku!