# CAST: Real-Time Motion Capture for Any Skeleton Topology

arXiv: https://arxiv.org/abs/2610.14856 · HTML: https://mlsanity.com/paper/2610.14856/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 22 of its 30 references that are in the map.
- First posted on arXiv 10 Oct 2026.

## Builds on (most cited first, 22 of 22)

- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [FiLM: Visual Reasoning with a General Conditioning Layer](https://mlsanity.com/paper/1709.07871/) (2017)
- [DINOv3](https://mlsanity.com/paper/2508.10104/) (2025)
- [On the Continuity of Rotation Representations in Neural Networks](https://mlsanity.com/paper/1812.07035/) (2018)
- [Animate Anyone: Consistent and Controllable Image-to-Video Synthesis for Character Animation](https://mlsanity.com/paper/2311.17117/) (2023)
- [WHAM: Reconstructing World-grounded Humans with Accurate 3D Motion](https://mlsanity.com/paper/2312.07531/) (2023)
- [World-Grounded Human Motion Recovery via Gravity-View Coordinates](https://mlsanity.com/paper/2409.06662/) (2024)
- [Skeleton-Aware Networks for Deep Motion Retargeting](https://mlsanity.com/paper/2005.05732/) (2020)
- [QuaterNet: A Quaternion-based Recurrent Model for Human Motion](https://mlsanity.com/paper/1805.06485/) (2018)
- [Neural Kinematic Networks for Unsupervised Motion Retargetting](https://mlsanity.com/paper/1804.05653/) (2018)
- [MIMO: Controllable Character Video Synthesis with Spatial Decomposed Modeling](https://mlsanity.com/paper/2409.16160/) (2024)
- [Puppeteer: Rig and Animate Your 3D Models](https://mlsanity.com/paper/2508.10898/) (2025)
- [AnyTop: Character Animation Diffusion with Any Topology](https://mlsanity.com/paper/2502.17327/) (2025)
- [Contact-Aware Retargeting of Skinned Motion](https://mlsanity.com/paper/2109.07431/) (2021)
- [Motion2Motion: Cross-topology Motion Transfer with Sparse Correspondence](https://mlsanity.com/paper/2508.13139/) (2025)
- [MoCapAnything: Unified 3D Motion Capture for Arbitrary Skeletons from Monocular Videos](https://mlsanity.com/paper/2512.10881/) (2025)
- [RoMo: A Robust Solver for Full-body Unlabeled Optical Motion Capture](https://mlsanity.com/paper/2410.02788/) (2024)
- [PoseAnything: Universal Pose-guided Video Generation with Part-aware Temporal Coherence](https://mlsanity.com/paper/2512.13465/) (2025)
- [STaR: Seamless Spatial-Temporal Aware Motion Retargeting with Penetration and Consistency Constraints](https://mlsanity.com/paper/2504.06504/) (2025)
- [MoCapAnything V2: End-to-End Motion Capture for Arbitrary Skeletons](https://mlsanity.com/paper/2604.28130/) (2026)
- [TopoCap: Learning Topology-Agnostic Motion Priors for Monocular Video-to-Animation](https://mlsanity.com/paper/2606.12153/) (2026)
- [SAMoR: Motion Modelling for Articulated Objects of Any Skeleton and Topology](https://mlsanity.com/paper/2607.02148/) (2026)

## Abstract (arXiv)

This paper studies the problem of recovering motion from monocular video, even when the target skeleton topology differs from the subject observed in the video. Although previous methods have made efforts to resolve this problem, they remain vulnerable to joint-rotation errors that are coupled across the skeleton and can propagate along its kinematic hierarchy. To mitigate these errors, we present CAST, a topology-aware motion capture framework that combines learned local rotations with geometric correction guided by predicted global bone directions. Our observability analysis characterizes which rotational components can be constrained by outgoing bone directions. Guided by this analysis, GRACE uses reliability-gated alignment to correct composed orientations in parent-to-child order, leaving unconstrained components to the learned rotation predictor. The motion predictor and correction module are trained jointly with motion-derived supervision, using a frozen image encoder. Experiments under the TopoCap evaluation protocol show that the proposed framework improves motion recovery across different target topologies, including unseen structures. Reference-based cross-skeleton evaluation on a curated Mixamo benchmark and a user preference study further assess transfer to different target structures. A causal CAST-B variant achieves over 51 FPS on a single NVIDIA A100 GPU, with timing covering image encoding and motion prediction. Code repository: https://github.com/fankewen/CAST.