# Look Ma, no markers: holistic performance capture without the hassle

arXiv: https://arxiv.org/abs/2410.11520 · HTML: https://mlsanity.com/paper/2410.11520/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 45 of its 80 references that are in the map.
- First posted on arXiv 15 Oct 2024.

## Builds on (most cited first, 30 of 45)

- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [Density estimation using Real NVP](https://mlsanity.com/paper/1605.08803/) (2016)
- [Expressive Body Capture: 3D Hands, Face, and Body from a Single Image](https://mlsanity.com/paper/1904.05866/) (2019)
- [Deep High-Resolution Representation Learning for Human Pose Estimation](https://mlsanity.com/paper/1902.09212/) (2019)
- [AMASS: Archive of Motion Capture as Surface Shapes](https://mlsanity.com/paper/1904.03278/) (2019)
- [On the Continuity of Rotation Representations in Neural Networks](https://mlsanity.com/paper/1812.07035/) (2018)
- [End-to-end Recovery of Human Shape and Pose](https://mlsanity.com/paper/1712.06584/) (2017)
- [Learning to Reconstruct 3D Human Pose and Shape via Model-fitting in the Loop](https://mlsanity.com/paper/1909.12828/) (2019)
- [End-to-End Human Pose and Mesh Reconstruction with Transformers](https://mlsanity.com/paper/2012.09760/) (2020)
- [GRAB: A Dataset of Whole-Body Human Grasping of Objects](https://mlsanity.com/paper/2008.11200/) (2020)
- [Humans in 4D: Reconstructing and Tracking Humans with Transformers](https://mlsanity.com/paper/2305.20091/) (2023)
- [HuMoR: 3D Human Motion Model for Robust Pose Estimation](https://mlsanity.com/paper/2105.04668/) (2021)
- [HybrIK: A Hybrid Analytical-Neural Inverse Kinematics Solution for 3D Human Pose and Shape Estimation](https://mlsanity.com/paper/2011.14672/) (2020)
- [I2L-MeshNet: Image-to-Lixel Prediction Network for Accurate 3D Human Pose and Mesh Estimation from a Single RGB Image](https://mlsanity.com/paper/2008.03713/) (2020)
- [FreiHAND: A Dataset for Markerless Capture of Hand Pose and Shape from Single RGB Images](https://mlsanity.com/paper/1909.04349/) (2019)
- [InterHand2.6M: A Dataset and Baseline for 3D Interacting Hand Pose Estimation from a Single RGB Image](https://mlsanity.com/paper/2008.09309/) (2020)
- [Total Capture: A 3D Deformation Model for Tracking Faces, Hands, and Bodies](https://mlsanity.com/paper/1801.01615/) (2018)
- [BEDLAM: A Synthetic Dataset of Bodies Exhibiting Detailed Lifelike Animated Motion](https://mlsanity.com/paper/2306.16940/) (2023)
- [WHAM: Reconstructing World-grounded Humans with Accurate 3D Motion](https://mlsanity.com/paper/2312.07531/) (2023)
- [Embodied Hands: Modeling and Capturing Hands and Bodies Together](https://mlsanity.com/paper/2201.02610/) (2022)
- [Learning to Regress 3D Face Shape and Expression from an Image without 3D Supervision](https://mlsanity.com/paper/1905.06817/) (2019)
- [FrankMocap: A Monocular 3D Whole-Body Pose Estimation System via Regression and Integration](https://mlsanity.com/paper/2108.06428/) (2021)
- [Towards Metrical Reconstruction of Human Faces](https://mlsanity.com/paper/2204.06607/) (2022)
- [Learnable Triangulation of Human Pose](https://mlsanity.com/paper/1905.05754/) (2019)
- [Probabilistic Modeling for Human Mesh Recovery](https://mlsanity.com/paper/2108.11944/) (2021)
- [PyMAF-X: Towards Well-aligned Full-body Model Regression from Monocular Images](https://mlsanity.com/paper/2207.06400/) (2022)
- [GOAL: Generating 4D Whole-Body Motion for Hand-Object Grasping](https://mlsanity.com/paper/2112.11454/) (2021)
- [Collaborative Regression of Expressive Bodies using Moderation](https://mlsanity.com/paper/2105.05301/) (2021)
- [One-Stage 3D Whole-Body Mesh Recovery with Component Aware Transformer](https://mlsanity.com/paper/2303.16160/) (2023)
- [Multi-HMR: Multi-Person Whole-Body Human Mesh Recovery in a Single Shot](https://mlsanity.com/paper/2402.14654/) (2024)

## Built on by (most cited first, 23 of 23)

- [SAM 3D Body: Robust Full-Body Human Mesh Recovery](https://mlsanity.com/paper/2602.15989/) (2026)
- [BEDLAM2.0: Synthetic Humans and Cameras in Motion](https://mlsanity.com/paper/2511.14394/) (2025)
- [Generative Zoo](https://mlsanity.com/paper/2412.08101/) (2024)
- [DAViD: Data-efficient and Accurate Vision Models from Synthetic Data](https://mlsanity.com/paper/2507.15365/) (2025)
- [NTIRE 2025 Challenge on Video Quality Enhancement for Video Conferencing: Datasets, Methods and Results](https://mlsanity.com/paper/2505.18988/) (2025)
- [From Frames to Sequences: Temporally Consistent Human-Centric Dense Prediction](https://mlsanity.com/paper/2602.01661/) (2026)
- [GASP: Gaussian Avatars with Synthetic Priors](https://mlsanity.com/paper/2412.07739/) (2024)
- [CondiMen: Conditional Multi-Person Mesh Recovery](https://mlsanity.com/paper/2412.13058/) (2024)
- [SEREP: Semantic Facial Expression Representation for Robust In-the-Wild Capture and Retargeting](https://mlsanity.com/paper/2412.14371/) (2024)
- [Kineo: Calibration-Free Metric Motion Capture From Sparse RGB Cameras](https://mlsanity.com/paper/2510.24464/) (2025)
- [SHOW3D: Capturing Scenes of 3D Hands and Objects in the Wild](https://mlsanity.com/paper/2603.28760/) (2026)
- [VoluMe -- Authentic 3D Video Calls from Live Gaussian Splat Prediction](https://mlsanity.com/paper/2507.21311/) (2025)
- [Bringing Your Portrait to 3D Presence](https://mlsanity.com/paper/2511.22553/) (2025)
- [PEAR: Pixel-aligned Expressive humAn mesh Recovery](https://mlsanity.com/paper/2601.22693/) (2026)
- [Bitrate-Controlled Diffusion for Disentangling Motion and Content in Video](https://mlsanity.com/paper/2509.08376/) (2025)
- [GNM Head: A Generative aNthropometric Model of the human head](https://mlsanity.com/paper/2607.23687/) (2026)
- [Eyelid Fold Consistency in Facial Modeling](https://mlsanity.com/paper/2410.13760/) (2024)
- [Bimanual 3D Hand Motion and Articulation Forecasting in Everyday Images](https://mlsanity.com/paper/2510.06145/) (2025)
- [MAMMA: Markerless & Automatic Multi-Person Motion Action Capture](https://mlsanity.com/paper/2506.13040/) (2025)
- [VOccl3D: A Video Benchmark Dataset for 3D Human Pose and Shape Estimation under real Occlusions](https://mlsanity.com/paper/2508.06757/) (2025)
- [A Multi-View Pipeline and Benchmark Dataset for 3D Hand Pose Estimation in Surgery](https://mlsanity.com/paper/2601.15918/) (2026)
- [Revisiting an Old Perspective Projection for Monocular 3D Morphable Models Regression](https://mlsanity.com/paper/2603.04958/) (2026)
- [RealDenseFace: Real-time Monocular 3D Face Reconstruction from Dense UV-space Priors](https://mlsanity.com/paper/2608.09238/) (2026)

## Abstract (arXiv)

We tackle the problem of highly-accurate, holistic performance capture for the face, body and hands simultaneously. Motion-capture technologies used in film and game production typically focus only on face, body or hand capture independently, involve complex and expensive hardware and a high degree of manual intervention from skilled operators. While machine-learning-based approaches exist to overcome these problems, they usually only support a single camera, often operate on a single part of the body, do not produce precise world-space results, and rarely generalize outside specific contexts. In this work, we introduce the first technique for marker-free, high-quality reconstruction of the complete human body, including eyes and tongue, without requiring any calibration, manual intervention or custom hardware. Our approach produces stable world-space results from arbitrary camera rigs as well as supporting varied capture environments and clothing. We achieve this through a hybrid approach that leverages machine learning models trained exclusively on synthetic data and powerful parametric models of human shape and motion. We evaluate our method on a number of body, face and hand reconstruction benchmarks and demonstrate state-of-the-art results that generalize on diverse datasets.