# Cross-Model Distillation of a Human-Pose Foundation Model from Unannotated Infant Video for Markerless 3D Pose Estimation

arXiv: https://arxiv.org/abs/2609.01840 · HTML: https://mlsanity.com/paper/2609.01840/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 35 references that are in the map.
- First posted on arXiv 1 Sep 2026.

## Builds on (most cited first, 18 of 18)

- [Expressive Body Capture: 3D Hands, Face, and Body from a Single Image](https://mlsanity.com/paper/1904.05866/) (2019)
- [SAM 3: Segment Anything with Concepts](https://mlsanity.com/paper/2511.16719/) (2025)
- [Sapiens: Foundation for Human Vision Models](https://mlsanity.com/paper/2408.12569/) (2024)
- [Equinox: neural networks in JAX via callable PyTrees and filtered transformations](https://mlsanity.com/paper/2111.00254/) (2021)
- [SAM 3D Body: Robust Full-Body Human Mesh Recovery](https://mlsanity.com/paper/2602.15989/) (2026)
- [Learning 3D Human Pose Estimation from Dozens of Datasets using a Geometry-Aware Autoencoder to Bridge Between Skeleton Formats](https://mlsanity.com/paper/2212.14474/) (2022)
- [Invariant Representation Learning for Infant Pose Estimation with Small Data](https://mlsanity.com/paper/2010.06100/) (2020)
- [Learning and Tracking the 3D Body Shape of Freely Moving Infants from RGB-D sequences](https://mlsanity.com/paper/1810.07538/) (2018)
- [Differentiable Biomechanics Unlocks Opportunities for Markerless Motion Capture](https://mlsanity.com/paper/2402.17192/) (2024)
- [Improved Trajectory Reconstruction for Markerless Pose Estimation](https://mlsanity.com/paper/2303.02413/) (2023)
- [On Triangulation as a Form of Self-Supervision for 3D Human Pose Estimation](https://mlsanity.com/paper/2203.15865/) (2022)
- [Human Mesh Modeling for Anny Body](https://mlsanity.com/paper/2511.03589/) (2025)
- [Automatic infant 2D pose estimation from videos: comparing seven deep neural network methods](https://mlsanity.com/paper/2406.17382/) (2024)
- [Monocular Biomechanical Tracking of Fingers with Inverse Kinematics to Foundation Models](https://mlsanity.com/paper/2605.09258/) (2026)
- [Sapiens2](https://mlsanity.com/paper/2604.21681/) (2026)
- [Leveraging Synthetic Adult Datasets for Unsupervised Infant Pose Estimation](https://mlsanity.com/paper/2504.05789/) (2025)
- [Markerless Motion Capture for Biomechanical Whole-Body Kinematic Estimation in Infants](https://mlsanity.com/paper/2605.17120/) (2026)
- [Biomechanical 3D Body: Self-Supervised Distillation of Biomechanical Pose from a 3D Body Foundation Model](https://mlsanity.com/paper/2608.29928/) (2026)

## Abstract (arXiv)

Spontaneous movement is one of the earliest windows onto an infant's neuromotor health, and structured clinical instruments that score it are validated early predictors of cerebral-palsy risk. However, they require specially trained raters, are time-consuming, and carry inter-rater variability. This motivates automated, video-based markerless assessment, especially as marker-based motion capture is impractical in infants. Yet the foundation models that make markerless capture possible are trained almost entirely on adults: our recent multi-view infant study found that no single model is jointly best, with strong 2D keypoint accuracy and direct 3D body recovery split across different models. While that study identifies this trade-off, it does not resolve it. Here, we perform cross-model distillation from the Sapiens 2 pose model into the SAM 3D Body model, using unannotated infant video alone. A frozen teacher supplies dense pseudo-labels, and a differentiable renderer aligns the predicted mesh to them in the training loop. On eleven held-out infants (18 sessions, 173 recordings) under our prior study's multi-view protocol, fine-tuning improves same-view 2D keypoint agreement with the Sapiens reference (median body percentage of correct keypoints @ 10px 0.22 -> 0.42, face 0.22 -> 0.42) and Procrustes-aligned mean per joint 3D position error (25.5 -> 22.2 mm). This demonstrates how cross-model distillation improves SAM 3D Body model performance on infants.