Evan LuebbertSoftware Engineer
EVAN LUEBBERTINHERITANCE
INHERITANCEEXPERIENCE

MACHINE LEARNING & MOTION CAPTURE ENGINEERING INTERN

Inheritance

FALL 2025

Motion Data for Machine Learning

At Inheritance, I built a Python and Blender pipeline that converted the entire AMASS research dataset into standardized skeletal animation for machine learning training in Unreal Engine. The pipeline processed 11,265 motions from 344 subjects, representing more than 65 hours of motion capture. The team estimated that around 90% could support training, saving roughly three months of data generation and $70,000 in equivalent motion capture production.

11,265
NEW MOTIONS AVAILABLE
3 MONTHS
DATA GENERATION SAVED
$70K
MOTION CAPTURE VALUE
A sample of retargeted motion capture animations.

During my internship, Inheritance operated under the name KikiTora. The company turned ordinary video into detailed human motion, with a longer-term goal of making recorded behavior available as structured data for machine learning.

The training workflow ran in reverse: known motion drove standardized characters in Unreal, where controlled camera, lighting, and rendering variations created many video-to-motion pairs. Compatible motion capture was the limiting factor, which made this pipeline especially valuable.

INHERITANCECHALLENGE

The Motion Data Bottleneck

Training a model to generate motion capture from video requires matched examples of both: a video input and the exact 3D motion behind it. Producing enough of these pairs at the scale required for machine learning is difficult and expensive.

Collecting video and motion pairs directly was not practical at the scale the model required. Instead, we started with known motion capture and used it to generate the matching video. The animations drove standardized characters in Unreal Engine environments, where the same performance could be rendered from different camera angles with different lighting, character meshes, image quality, and levels of occlusion. Every render remained paired with the original animation, giving the model both its visual input and the correct 3D motion.

This may seem backwards when our goal is to turn real video into motion capture. However, starting from motion solved the problem of obtaining an exact label for every frame. It also made video variation effectively unlimited. One performance could produce countless training examples, but every example still depended on the same underlying movement. The number and diversity of available motion capture performances remained the main constraint.

At the time, Inheritance produced much of its own training motion capture. This was a costly operation requiring a capture stage, performers, and production staff. Existing animation data is difficult to license for machine learning, while new capture sessions are expensive and time-consuming. Expanding the motion library meant either producing more performances or finding another source the company could use.

AMASS, short for Archive of Motion Capture as Surface Shapes, offered another source. It is a large research archive that combines motion capture from many academic datasets into a shared human-body format. The archive contained more than 65 hours of motion across 11,265 animations and 344 subjects.

The as Surface Shapes part of the name captures the compatibility problem. AMASS represented motion through SMPL-H body-model parameters that produce a rigged human surface mesh. Inheritance needed conventional animation keyed onto its exact production skeleton, ready to move through Blender, GLB, and Unreal. In other words, the source was organized around a deformable body surface and its parameters, while the destination was organized around animation keys on a specific skeleton. The motion had to be reconstructed in 3D, aligned with the target hierarchy, rest pose, proportions, and coordinate spaces, and exported in the required format.

THE MOTION AND BODY DIVERSITY REPRESENTED IN AMASS.
INHERITANCEENGINEERING

Rebuilding Motion in 3D

AMASS offers thousands of motions, but making them usable required a complex engineering solution. I built an automated Python and Blender pipeline that reconstructed each sequence from SMPL-H parameters, transferred it onto Inheritance's fixed production armature, and exported consistent GLB animations for Unreal Engine.

Ordinary retargeting starts with two complete skeletons. Here, the source files did not contain a conventional keyed armature. They stored joint rotations and root movement inside a parametric human-body mesh model. The pipeline first had to recover where every joint moved in three dimensions, then turn those trajectories into animation on a skeleton with a different rest pose, proportions, hierarchy, and coordinate spaces.

AMASS MOTION, REBUILT ON A PRODUCTION ARMATURE.

TECHNOLOGY

  • Python
  • Blender
  • NumPy
  • SMPL-H
  • glTF / GLB
  • Unreal Engine

CORE PIPELINE

Each frame contained rotations for 52 SMPL-H joints plus the movement of the root joint. I used forward kinematics to reconstruct the body's world-space joint positions through the parent-child hierarchy. Those positions then drove Inheritance's standardized armature in Blender, where the evaluated motion was baked into keyframes and exported as an animated GLB.

The pipeline for each animation:

  1. Read the joint rotations, root movement, and source frame rate.
  2. Reconstructed every joint's position through the SMPL-H hierarchy.
  3. Aligned the motion with the target armature and coordinate system.
  4. Drove and baked the standardized skeleton in Blender.
  5. Exported consistent animated GLBs for the Unreal ML workflow.

ENGINEERING HIGHLIGHTS

RECONCILING REST POSES

AMASS baked in the SMPL-H template T-pose, while the production pipeline depended on a standardized base A-pose for calibration, mesh weight painting, and other downstream character setup. I calculated the transforms required to convert the motion from the source pose into the production pose so every animation used the same armature without changing the intended motion.

BALANCING PROPORTIONS AND MOTION

The reconstructed joint paths came from parametric bodies whose proportions did not match the production rig. Forcing every target bone to reach those positions exactly caused the skeleton to stretch, preserving the coordinates but distorting the body and the character of the movement. The solve was to treat the joint paths as motion guides rather than literal bone dimensions, then use Blender constraints and baking to transfer their timing and trajectories onto the standardized armature. This preserved the production skeleton's proportions while retaining the recognizable intent of each performance.

TRANSLATING THREE ROTATION SYSTEMS

The motion had to pass through three mathematical representations because each one served a different part of the pipeline. AMASS stored each joint rotation in axis-angle form, a compact three-value encoding suited to a large dataset but not to composing an entire skeletal hierarchy. I converted those values into rotation matrices with Rodrigues' formula so parent and child transforms could be combined through forward kinematics and evaluated across local and world coordinate spaces. The baked animation ultimately entered Blender and GLB's quaternion-based representation, which supports stable rotation and interpolation in animation tools.

Although all three representations encode orientation, they are not interchangeable without conversion. Every rotation had to remain in the correct coordinate frame as it moved from compact source data, through hierarchical calculations, and into the exported animation. Errors could turn joints around the wrong axis, reverse a limb, or allow small errors to accumulate through the hierarchy.

INHERITANCEIMPACT

Accelerating the Data Roadmap

The AMASS retargeting pipeline gave Inheritance a large, usable motion library without requiring equivalent new motion capture sessions. It processed all 11,265 animations available to the company, with approximately 90% qualifying as useful for training. The pipeline provided thousands of performances across 344 subjects and greatly expanded the team's capacity to generate paired video and motion for training.

Astrid Wilde, CEO of Inheritance, estimated that the pipeline accelerated the company's data-generation roadmap by approximately three months and avoided roughly $70,000 in equivalent stage, performer, and production costs. It also broadened the foundation of human motion available for controlled training renders.

FROM MOTION CAPTURE TO PHYSICAL AI

At KikiTora, the immediate goal was to recover detailed human motion from ordinary video. Inheritance now frames that idea more broadly, converting video into structured representations of behavior, contact, and action that robots and world models can use for training. The opportunity extends beyond animation. For computer vision, it means moving from recognizing what appears in a frame toward recovering how people move and interact with the physical world over time.

At scale, that approach could allow robotics teams to draw from recorded human behavior instead of recreating every demonstration for each task, environment, or machine. A growing body of structured examples could help models generalize across a wider range of conditions.

TURNING RESEARCH INTO IMPACT

Developing the AMASS retargeting pipeline was especially rewarding because it tangibly accelerated the mission by introducing thousands of motions into the ML training workflow. My work addressed a serious production constraint and gave new life to data beyond its original research setting.

I was able to draw from several areas of my experience, from data infrastructure and 3D math to computer vision and Blender automation. The project brought together model training, computer graphics, and animation production, and I enjoyed figuring out how to make those systems work together. The result was a reliable pipeline that delivered measurable value to Inheritance while pointing to a much larger opportunity for structured motion data across computer vision, animation, robotics, and physical AI.