# KinTwin: Imitation Learning with Torque and Muscle Driven Biomechanical Models Enables Precise Replication of Able-Bodied and Impaired Movement from Markerless Motion Capture

arXiv: https://arxiv.org/abs/2505.13436 · HTML: https://mlsanity.com/paper/2505.13436/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Real-world robot offline RL, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 21 of its 36 references that are in the map.
- First posted on arXiv 19 May 2025.

## Placed in

- Offline RL and imitation learning from data / [Real-world robot offline RL](https://mlsanity.com/offlinerl/real-world-robot-offline-rl/)

## Builds on (most cited first, 21 of 21)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [AMASS: Archive of Motion Capture as Surface Shapes](https://mlsanity.com/paper/1904.03278/) (2019)
- [Brax -- A Differentiable Physics Engine for Large Scale Rigid Body Simulation](https://mlsanity.com/paper/2106.13281/) (2021)
- [Perpetual Humanoid Control for Real-time Simulated Avatars](https://mlsanity.com/paper/2305.06456/) (2023)
- [DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skills](https://mlsanity.com/paper/1804.02717/) (2018)
- [Universal Humanoid Motion Representations for Physics-Based Control](https://mlsanity.com/paper/2310.04582/) (2023)
- [SimPoE: Simulated Character Control for 3D Human Pose Estimation](https://mlsanity.com/paper/2104.00683/) (2021)
- [Accelerated Policy Learning with Parallel Differentiable Simulation](https://mlsanity.com/paper/2204.07137/) (2022)
- [CALM: Conditional Adversarial Latent Models for Directable Virtual Characters](https://mlsanity.com/paper/2305.02195/) (2023)
- [MyoSuite -- A contact-rich simulation suite for musculoskeletal motor control](https://mlsanity.com/paper/2205.13600/) (2022)
- [Neural Monocular 3D Human Motion Capture with Physical Awareness](https://mlsanity.com/paper/2105.01057/) (2021)
- [Residual Force Control for Agile Human Behavior Imitation and Extended Motion Synthesis](https://mlsanity.com/paper/2006.07364/) (2020)
- [MoVi: A Large Multipurpose Motion and Video Dataset](https://mlsanity.com/paper/2003.01888/) (2020)
- [PhysCap: Physically Plausible Monocular 3D Motion Capture in Real Time](https://mlsanity.com/paper/2008.08880/) (2020)
- [Learning 3D Human Pose Estimation from Dozens of Datasets using a Geometry-Aware Autoencoder to Bridge Between Skeleton Formats](https://mlsanity.com/paper/2212.14474/) (2022)
- [LocoMuJoCo: A Comprehensive Imitation Learning Benchmark for Locomotion](https://mlsanity.com/paper/2311.02496/) (2023)
- [MotioNet: 3D Human Motion Reconstruction from Monocular Video with Skeleton Consistency](https://mlsanity.com/paper/2006.12075/) (2020)
- [Adaptive Horizon Actor-Critic for Policy Learning in Contact-Rich Differentiable Simulation](https://mlsanity.com/paper/2405.17784/) (2024)
- [Differentiable Biomechanics Unlocks Opportunities for Markerless Motion Capture](https://mlsanity.com/paper/2402.17192/) (2024)
- [Fusing uncalibrated IMUs and handheld smartphone video to reconstruct knee kinematics](https://mlsanity.com/paper/2405.17368/) (2024)
- [Advancing Monocular Video-Based Gait Analysis Using Motion Imitation with Physics-Based Simulation](https://mlsanity.com/paper/2402.12676/) (2024)

## Built on by (most cited first, 5 of 5)

- [Portable Biomechanics Laboratory: Clinically Accessible Movement Analysis from a Handheld Smartphone](https://mlsanity.com/paper/2507.08268/) (2025)
- [MIMIC-MJX: Neuromechanical Emulation of Animal Behavior](https://mlsanity.com/paper/2511.20532/) (2025)
- [Muscle Synergy Priors Enhance Biomechanical Fidelity in Predictive Musculoskeletal Locomotion Simulation](https://mlsanity.com/paper/2603.10474/) (2026)
- [BiomechAgent: AI-Assisted Biomechanical Analysis Through Code-Generating Agents](https://mlsanity.com/paper/2602.06975/) (2026)
- [Imitation Learning from Human Motion Alone Does Not Guarantee Biomechanically Plausible Gait Kinetics](https://mlsanity.com/paper/2603.12408/) (2026)

## Abstract (arXiv)

Broader access to high-quality movement analysis could greatly benefit movement science and rehabilitation, such as allowing more detailed characterization of movement impairments and responses to interventions, or even enabling early detection of new neurological conditions or fall risk. While emerging technologies are making it easier to capture kinematics with biomechanical models, or how joint angles change over time, inferring the underlying physics that give rise to these movements, including ground reaction forces, joint torques, or even muscle activations, is still challenging. Here we explore whether imitation learning applied to a biomechanical model from a large dataset of movements from able-bodied and impaired individuals can learn to compute these inverse dynamics. Although imitation learning in human pose estimation has seen great interest in recent years, our work differences in several ways: we focus on using an accurate biomechanical model instead of models adopted for computer vision, we test it on a dataset that contains participants with impaired movements, we reported detailed tracking metrics relevant for the clinical measurement of movement including joint angles and ground contact events, and finally we apply imitation learning to a muscle-driven neuromusculoskeletal model. We show that our imitation learning policy, KinTwin, can accurately replicate the kinematics of a wide range of movements, including those with assistive devices or therapist assistance, and that it can infer clinically meaningful differences in joint torques and muscle activations. Our work demonstrates the potential for using imitation learning to enable high-quality movement analysis in clinical practice.