# Differentiable Biomechanics for Markerless Motion Capture in Upper Limb Stroke Rehabilitation: A Comparison with Optical Motion Capture

arXiv: https://arxiv.org/abs/2411.14992 · HTML: https://mlsanity.com/paper/2411.14992/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 44 references that are in the map.
- First posted on arXiv 22 Nov 2024.

## Builds on (most cited first, 8 of 8)

- [Markerless tracking of user-defined features with deep learning](https://mlsanity.com/paper/1804.03142/) (2018)
- [MyoSuite -- A contact-rich simulation suite for musculoskeletal motor control](https://mlsanity.com/paper/2205.13600/) (2022)
- [MoVi: A Large Multipurpose Motion and Video Dataset](https://mlsanity.com/paper/2003.01888/) (2020)
- [Learning 3D Human Pose Estimation from Dozens of Datasets using a Geometry-Aware Autoencoder to Bridge Between Skeleton Formats](https://mlsanity.com/paper/2212.14474/) (2022)
- [LocoMuJoCo: A Comprehensive Imitation Learning Benchmark for Locomotion](https://mlsanity.com/paper/2311.02496/) (2023)
- [Differentiable Biomechanics Unlocks Opportunities for Markerless Motion Capture](https://mlsanity.com/paper/2402.17192/) (2024)
- [A Review of Computational Approaches for Evaluation of Rehabilitation Exercises](https://mlsanity.com/paper/2003.08767/) (2020)
- [Fusing uncalibrated IMUs and handheld smartphone video to reconstruct knee kinematics](https://mlsanity.com/paper/2405.17368/) (2024)

## Built on by (most cited first, 8 of 8)

- [Portable Biomechanics Laboratory: Clinically Accessible Movement Analysis from a Handheld Smartphone](https://mlsanity.com/paper/2507.08268/) (2025)
- [Monocular Biomechanical Tracking of Fingers with Inverse Kinematics to Foundation Models](https://mlsanity.com/paper/2605.09258/) (2026)
- [Calibrated Uncertainty for Trustworthy Clinical Gait Analysis Using Probabilistic Multiview Markerless Motion Capture](https://mlsanity.com/paper/2601.22412/) (2026)
- [MIMIC-MJX: Neuromechanical Emulation of Animal Behavior](https://mlsanity.com/paper/2511.20532/) (2025)
- [Monocular Markerless Motion Capture Enables Quantitative Assessment of Upper Extremity Reachable Workspace](https://mlsanity.com/paper/2602.13176/) (2026)
- [Biomechanical 3D Body: Self-Supervised Distillation of Biomechanical Pose from a 3D Body Foundation Model](https://mlsanity.com/paper/2608.29928/) (2026)
- [Biomechanics-aware Multi-view Markerless Motion Capture of Dexterous Hand Movements](https://mlsanity.com/paper/2607.02796/) (2026)
- [Markerless Motion Capture in Routine Clinical Upper Limb Assessments: Validity and Insights Beyond Ordinal Scoring](https://mlsanity.com/paper/2607.23608/) (2026)

## Abstract (arXiv)

Marker-based Optical Motion Capture (OMC) paired with biomechanical modeling is currently considered the most precise and accurate method for measuring human movement kinematics. However, combining differentiable biomechanical modeling with Markerless Motion Capture (MMC) offers a promising approach to motion capture in clinical settings, requiring only minimal equipment, such as synchronized webcams, and minimal effort for data collection. This study compares key kinematic outcomes from biomechanically modeled MMC and OMC data in 15 stroke patients performing the drinking task, a functional task recommended for assessing upper limb movement quality. We observed a high level of agreement in kinematic trajectories between MMC and OMC, as indicated by high correlations (median r above 0.95 for the majority of kinematic trajectories) and median RMSE values ranging from 2-5 degrees for joint angles, 0.04 m/s for end-effector velocity, and 6 mm for trunk displacement. Trial-to-trial biases between OMC and MMC were consistent within participant sessions, with interquartile ranges of bias around 1-3 degrees for joint angles, 0.01 m/s in end-effector velocity, and approximately 3mm for trunk displacement. Our findings indicate that our MMC for arm tracking is approaching the accuracy of marker-based methods, supporting its potential for use in clinical settings. MMC could provide valuable insights into movement rehabilitation in stroke patients, potentially enhancing the effectiveness of rehabilitation strategies.