# MoCapAnything: Unified 3D Motion Capture for Arbitrary Skeletons from Monocular Videos

arXiv: https://arxiv.org/abs/2512.10881 · HTML: https://mlsanity.com/paper/2512.10881/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 44 of its 53 references that are in the map.
- First posted on arXiv 11 Dec 2025.

## Builds on (most cited first, 30 of 44)

- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Expressive Body Capture: 3D Hands, Face, and Body from a Single Image](https://mlsanity.com/paper/1904.05866/) (2019)
- [Objaverse: A Universe of Annotated 3D Objects](https://mlsanity.com/paper/2212.08051/) (2022)
- [Deep High-Resolution Representation Learning for Human Pose Estimation](https://mlsanity.com/paper/1902.09212/) (2019)
- [End-to-end Recovery of Human Shape and Pose](https://mlsanity.com/paper/1712.06584/) (2017)
- [Objaverse-XL: A Universe of 10M+ 3D Objects](https://mlsanity.com/paper/2307.05663/) (2023)
- [VIBE: Video Inference for Human Body Pose and Shape Estimation](https://mlsanity.com/paper/1912.05656/) (2019)
- [Simple Baselines for Human Pose Estimation and Tracking](https://mlsanity.com/paper/1804.06208/) (2018)
- [Humans in 4D: Reconstructing and Tracking Humans with Transformers](https://mlsanity.com/paper/2305.20091/) (2023)
- [Cascaded Pyramid Network for Multi-Person Pose Estimation](https://mlsanity.com/paper/1711.07319/) (2017)
- [Learning Category-Specific Mesh Reconstruction from Image Collections](https://mlsanity.com/paper/1803.07549/) (2018)
- [3D Menagerie: Modeling the 3D shape and pose of animals](https://mlsanity.com/paper/1611.07700/) (2016)
- [DeepPose: Human Pose Estimation via Deep Neural Networks](https://mlsanity.com/paper/1312.4659/) (2013)
- [HigherHRNet: Scale-Aware Representation Learning for Bottom-Up Human Pose Estimation](https://mlsanity.com/paper/1908.10357/) (2019)
- [WHAM: Reconstructing World-grounded Humans with Accurate 3D Motion](https://mlsanity.com/paper/2312.07531/) (2023)
- [BANMo: Building Animatable 3D Neural Models from Many Casual Videos](https://mlsanity.com/paper/2112.12761/) (2021)
- [Beyond Static Features for Temporally Consistent 3D Human Pose and Shape from a Video](https://mlsanity.com/paper/2011.08627/) (2020)
- [GLAMR: Global Occlusion-Aware Human Mesh Recovery with Dynamic Cameras](https://mlsanity.com/paper/2112.01524/) (2021)
- [Human Pose Regression with Residual Log-likelihood Estimation](https://mlsanity.com/paper/2107.11291/) (2021)
- [One-Stage 3D Whole-Body Mesh Recovery with Component Aware Transformer](https://mlsanity.com/paper/2303.16160/) (2023)
- [LASR: Learning Articulated Shape Reconstruction from a Monocular Video](https://mlsanity.com/paper/2105.02976/) (2021)
- [MagicPony: Learning Articulated 3D Animals in the Wild](https://mlsanity.com/paper/2211.12497/) (2022)
- [TRAM: Global Trajectory and Motion of 3D Humans from in-the-wild Videos](https://mlsanity.com/paper/2403.17346/) (2024)
- [Multi-HMR: Multi-Person Whole-Body Human Mesh Recovery in a Single Shot](https://mlsanity.com/paper/2402.14654/) (2024)
- [PoseAug: A Differentiable Pose Augmentation Framework for 3D Human Pose Estimation](https://mlsanity.com/paper/2105.02465/) (2021)
- [SimCC: a Simple Coordinate Classification Perspective for Human Pose Estimation](https://mlsanity.com/paper/2107.03332/) (2021)
- [LASSIE: Learning Articulated Shapes from Sparse Image Ensemble via 3D Part Discovery](https://mlsanity.com/paper/2207.03434/) (2022)
- [Learning the 3D Fauna of the Web](https://mlsanity.com/paper/2401.02400/) (2024)
- [Explicit Box Detection Unifies End-to-End Multi-Person Pose Estimation](https://mlsanity.com/paper/2302.01593/) (2023)

## Built on by (most cited first, 10 of 10)

- [MoCapAnything V2: End-to-End Motion Capture for Arbitrary Skeletons](https://mlsanity.com/paper/2604.28130/) (2026)
- [MorphGS: Morphology-Adaptive Articulated 3D Motion Transfer from Videos](https://mlsanity.com/paper/2601.02716/) (2026)
- [A Unified Conditional Flow for Motion Generation, Editing, and Intra-Structural Retargeting](https://mlsanity.com/paper/2604.13427/) (2026)
- [Follow Your Track: Precise Skeleton Animation Controlled by 3D Trajectories](https://mlsanity.com/paper/2606.25344/) (2026)
- [SkelGen4D: Weakly-Supervised Skeleton-Based 4D Generation for Text-Driven Mesh Animation](https://mlsanity.com/paper/2607.08246/) (2026)
- [UniMate: One Unified Model to Animate Diverse Skeletons](https://mlsanity.com/paper/2609.05415/) (2026)
- [R-DMesh: Video-Guided 3D Animation via Rectified Dynamic Mesh Flow](https://mlsanity.com/paper/2605.13838/) (2026)
- [AnyAct: Towards Human Reenactment of Character Motion From Video](https://mlsanity.com/paper/2605.15497/) (2026)
- [SkelMo: Universal Skeletal Motion Generation for 3D Rigged Shapes](https://mlsanity.com/paper/2606.01518/) (2026)
- [CAST: Real-Time Motion Capture for Any Skeleton Topology](https://mlsanity.com/paper/2610.14856/) (2026)

## Abstract (arXiv)

Motion capture now underpins content creation far beyond digital humans, yet most existing pipelines remain species- or template-specific. We formalize this gap as Category-Agnostic Motion Capture (CAMoCap): given a monocular video and an arbitrary rigged 3D asset as a prompt, the goal is to reconstruct a rotation-based animation such as BVH that directly drives the specific asset. We present MoCapAnything, a reference-guided, factorized framework that first predicts 3D joint trajectories and then recovers asset-specific rotations via constraint-aware inverse kinematics. The system contains three learnable modules and a lightweight IK stage: (1) a Reference Prompt Encoder that extracts per-joint queries from the asset's skeleton, mesh, and rendered images; (2) a Video Feature Extractor that computes dense visual descriptors and reconstructs a coarse 4D deforming mesh to bridge the gap between video and joint space; and (3) a Unified Motion Decoder that fuses these cues to produce temporally coherent trajectories. We also curate Truebones Zoo with 1038 motion clips, each providing a standardized skeleton-mesh-render triad. Experiments on both in-domain benchmarks and in-the-wild videos show that MoCapAnything delivers high-quality skeletal animations and exhibits meaningful cross-species retargeting across heterogeneous rigs, enabling scalable, prompt-driven 3D motion capture for arbitrary assets. Project page: https://animotionlab.github.io/MoCapAnything/