# WHAM: Reconstructing World-grounded Humans with Accurate 3D Motion

arXiv: https://arxiv.org/abs/2312.07531 · HTML: https://mlsanity.com/paper/2312.07531/

## Key facts

- Cited by 206 papers in the map (data through 14 Oct 2026).
- Builds on 49 of its 60 references that are in the map.
- First posted on arXiv 12 Dec 2023.

## Builds on (most cited first, 30 of 49)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Expressive Body Capture: 3D Hands, Face, and Body from a Single Image](https://mlsanity.com/paper/1904.05866/) (2019)
- [AMASS: Archive of Motion Capture as Surface Shapes](https://mlsanity.com/paper/1904.03278/) (2019)
- [OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1812.08008/) (2018)
- [End-to-end Recovery of Human Shape and Pose](https://mlsanity.com/paper/1712.06584/) (2017)
- [Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Image](https://mlsanity.com/paper/1607.08128/) (2016)
- [DROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and RGB-D Cameras](https://mlsanity.com/paper/2108.10869/) (2021)
- [Learning to Reconstruct 3D Human Pose and Shape via Model-fitting in the Loop](https://mlsanity.com/paper/1909.12828/) (2019)
- [VIBE: Video Inference for Human Body Pose and Shape Estimation](https://mlsanity.com/paper/1912.05656/) (2019)
- [Monocular 3D Human Pose Estimation In The Wild Using Improved CNN Supervision](https://mlsanity.com/paper/1611.09813/) (2016)
- [ViTPose: Simple Vision Transformer Baselines for Human Pose Estimation](https://mlsanity.com/paper/2204.12484/) (2022)
- [Humans in 4D: Reconstructing and Tracking Humans with Transformers](https://mlsanity.com/paper/2305.20091/) (2023)
- [PARE: Part Attention Regressor for 3D Human Body Estimation](https://mlsanity.com/paper/2104.08527/) (2021)
- [HuMoR: 3D Human Motion Model for Robust Pose Estimation](https://mlsanity.com/paper/2105.04668/) (2021)
- [Learning 3D Human Dynamics from Video](https://mlsanity.com/paper/1812.01601/) (2018)
- [HybrIK: A Hybrid Analytical-Neural Inverse Kinematics Solution for 3D Human Pose and Shape Estimation](https://mlsanity.com/paper/2011.14672/) (2020)
- [I2L-MeshNet: Image-to-Lixel Prediction Network for Accurate 3D Human Pose and Mesh Estimation from a Single RGB Image](https://mlsanity.com/paper/2008.03713/) (2020)
- [Convolutional Mesh Regression for Single-Image Human Shape Reconstruction](https://mlsanity.com/paper/1905.03244/) (2019)
- [Pose2Mesh: Graph Convolutional Network for 3D Human Pose and Mesh Recovery from a 2D Human Pose](https://mlsanity.com/paper/2008.09047/) (2020)
- [CLIFF: Carrying Location Information in Full Frames into Human Pose and Shape Estimation](https://mlsanity.com/paper/2208.00571/) (2022)
- [BEDLAM: A Synthetic Dataset of Bodies Exhibiting Detailed Lifelike Animated Motion](https://mlsanity.com/paper/2306.16940/) (2023)
- [Learning to Estimate 3D Human Pose and Shape from a Single Color Image](https://mlsanity.com/paper/1805.04092/) (2018)
- [STAR: Sparse Trained Articulated Human Body Regressor](https://mlsanity.com/paper/2008.08535/) (2020)
- [MotionBERT: A Unified Perspective on Learning Human Motion Representations](https://mlsanity.com/paper/2210.06551/) (2022)
- [Deep Patch Visual Odometry](https://mlsanity.com/paper/2208.04726/) (2022)
- [Capturing and Inferring Dense Full-Body Human-Scene Contact](https://mlsanity.com/paper/2206.09553/) (2022)
- [Distribution-Aware Coordinate Representation for Human Pose Estimation](https://mlsanity.com/paper/1910.06278/) (2019)
- [Beyond Static Features for Temporally Consistent 3D Human Pose and Shape from a Video](https://mlsanity.com/paper/2011.08627/) (2020)
- [SMPLer-X: Scaling Up Expressive Human Pose and Shape Estimation](https://mlsanity.com/paper/2309.17448/) (2023)
- [GLAMR: Global Occlusion-Aware Human Mesh Recovery with Dynamic Cameras](https://mlsanity.com/paper/2112.01524/) (2021)

## Built on by (most cited first, 30 of 206)

- [HumanPlus: Humanoid Shadowing and Imitation from Humans](https://mlsanity.com/paper/2406.10454/) (2024)
- [World-Grounded Human Motion Recovery via Gravity-View Coordinates](https://mlsanity.com/paper/2409.06662/) (2024)
- [WiLoR: End-to-end 3D Hand Localization and Reconstruction in-the-wild](https://mlsanity.com/paper/2409.12259/) (2024)
- [TRAM: Global Trajectory and Motion of 3D Humans from in-the-wild Videos](https://mlsanity.com/paper/2403.17346/) (2024)
- [TWIST: Teleoperated Whole-Body Imitation System](https://mlsanity.com/paper/2505.02833/) (2025)
- [Geometry Forcing: Marrying Video Diffusion and 3D Representation for Consistent World Modeling](https://mlsanity.com/paper/2507.07982/) (2025)
- [PromptHMR: Promptable Human Mesh Recovery](https://mlsanity.com/paper/2504.06397/) (2025)
- [Deep Patch Visual SLAM](https://mlsanity.com/paper/2408.01654/) (2024)
- [SAM 3D Body: Robust Full-Body Human Mesh Recovery](https://mlsanity.com/paper/2602.15989/) (2026)
- [Visual Imitation Enables Contextual Humanoid Control](https://mlsanity.com/paper/2505.03729/) (2025)
- [HaWoR: World-Space Hand Motion Reconstruction from Egocentric Videos](https://mlsanity.com/paper/2501.02973/) (2025)
- [CameraHMR: Aligning People with Perspective](https://mlsanity.com/paper/2411.08128/) (2024)
- [Neural Localizer Fields for Continuous 3D Human Pose and Shape Estimation](https://mlsanity.com/paper/2407.07532/) (2024)
- [Dyn-HaMR: Recovering 4D Interacting Hand Motion from a Dynamic Camera](https://mlsanity.com/paper/2412.12861/) (2024)
- [GENMO: A GENeralist Model for Human MOtion](https://mlsanity.com/paper/2505.01425/) (2025)
- [Human3R: Everyone Everywhere All at Once](https://mlsanity.com/paper/2510.06219/) (2025)
- [Harmony4D: A Video Dataset for In-The-Wild Close Human Interactions](https://mlsanity.com/paper/2410.20294/) (2024)
- [Reconstructing People, Places, and Cameras](https://mlsanity.com/paper/2412.17806/) (2024)
- [WHAC: World-grounded Humans and Cameras](https://mlsanity.com/paper/2403.12959/) (2024)
- [Joint Optimization for 4D Human-Scene Reconstruction in the Wild](https://mlsanity.com/paper/2501.02158/) (2025)
- [PhysPT: Physics-aware Pretrained Transformer for Estimating Human Dynamics from Monocular Videos](https://mlsanity.com/paper/2404.04430/) (2024)
- [HuB: Learning Extreme Humanoid Balance](https://mlsanity.com/paper/2505.07294/) (2025)
- [ZeroHSI: Zero-Shot 4D Human-Scene Interaction by Video Generation](https://mlsanity.com/paper/2412.18600/) (2024)
- [BEDLAM2.0: Synthetic Humans and Cameras in Motion](https://mlsanity.com/paper/2511.14394/) (2025)
- [Look Ma, no markers: holistic performance capture without the hassle](https://mlsanity.com/paper/2410.11520/) (2024)
- [MultiWorld: Scalable Multi-Agent Multi-View Video World Models](https://mlsanity.com/paper/2604.18564/) (2026)
- [Scaling Large Motion Models with Million-Level Human Motions](https://mlsanity.com/paper/2410.03311/) (2024)
- [Predicting 4D Hand Trajectory from Monocular Videos](https://mlsanity.com/paper/2501.08329/) (2025)
- [AniMer: Animal Pose and Shape Estimation Using Family Aware Transformer](https://mlsanity.com/paper/2412.00837/) (2024)
- [COIN: Control-Inpainting Diffusion Prior for Human and Camera Motion Estimation](https://mlsanity.com/paper/2408.16426/) (2024)

## Abstract (arXiv)

The estimation of 3D human motion from video has progressed rapidly but current methods still have several key limitations. First, most methods estimate the human in camera coordinates. Second, prior work on estimating humans in global coordinates often assumes a flat ground plane and produces foot sliding. Third, the most accurate methods rely on computationally expensive optimization pipelines, limiting their use to offline applications. Finally, existing video-based methods are surprisingly less accurate than single-frame methods. We address these limitations with WHAM (World-grounded Humans with Accurate Motion), which accurately and efficiently reconstructs 3D human motion in a global coordinate system from video. WHAM learns to lift 2D keypoint sequences to 3D using motion capture data and fuses this with video features, integrating motion context and visual information. WHAM exploits camera angular velocity estimated from a SLAM method together with human motion to estimate the body's global trajectory. We combine this with a contact-aware trajectory refinement method that lets WHAM capture human motion in diverse conditions, such as climbing stairs. WHAM outperforms all existing 3D human motion recovery methods across multiple in-the-wild benchmarks. Code will be available for research purposes at http://wham.is.tue.mpg.de/