# RoHM: Robust Human Motion Reconstruction via Diffusion

arXiv: https://arxiv.org/abs/2401.08570 · HTML: https://mlsanity.com/paper/2401.08570/

## Key facts

- Cited by 48 papers in the map (data through 14 Oct 2026).
- Builds on 83 of its 103 references that are in the map.
- First posted on arXiv 16 Jan 2024.

## Builds on (most cited first, 30 of 83)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [Expressive Body Capture: 3D Hands, Face, and Body from a Single Image](https://mlsanity.com/paper/1904.05866/) (2019)
- [AMASS: Archive of Motion Capture as Surface Shapes](https://mlsanity.com/paper/1904.03278/) (2019)
- [On the Continuity of Rotation Representations in Neural Networks](https://mlsanity.com/paper/1812.07035/) (2018)
- [Human Motion Diffusion Model](https://mlsanity.com/paper/2209.14916/) (2022)
- [Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1611.08050/) (2016)
- [Planning with Diffusion for Flexible Behavior Synthesis](https://mlsanity.com/paper/2205.09991/) (2022)
- [End-to-end Recovery of Human Shape and Pose](https://mlsanity.com/paper/1712.06584/) (2017)
- [Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Image](https://mlsanity.com/paper/1607.08128/) (2016)
- [Learning to Reconstruct 3D Human Pose and Shape via Model-fitting in the Loop](https://mlsanity.com/paper/1909.12828/) (2019)
- [VIBE: Video Inference for Human Body Pose and Shape Estimation](https://mlsanity.com/paper/1912.05656/) (2019)
- [Executing your Commands via Motion Diffusion in Latent Space](https://mlsanity.com/paper/2212.04048/) (2022)
- [MotionGPT: Human Motion as a Foreign Language](https://mlsanity.com/paper/2306.14795/) (2023)
- [End-to-End Human Pose and Mesh Reconstruction with Transformers](https://mlsanity.com/paper/2012.09760/) (2020)
- [PhysDiff: Physics-Guided Human Motion Diffusion Model](https://mlsanity.com/paper/2212.02500/) (2022)
- [Human Motion Diffusion as a Generative Prior](https://mlsanity.com/paper/2303.01418/) (2023)
- [Humans in 4D: Reconstructing and Tracking Humans with Transformers](https://mlsanity.com/paper/2305.20091/) (2023)
- [PARE: Part Attention Regressor for 3D Human Body Estimation](https://mlsanity.com/paper/2104.08527/) (2021)
- [HuMoR: 3D Human Motion Model for Robust Pose Estimation](https://mlsanity.com/paper/2105.04668/) (2021)
- [Learning 3D Human Dynamics from Video](https://mlsanity.com/paper/1812.01601/) (2018)
- [HybrIK: A Hybrid Analytical-Neural Inverse Kinematics Solution for 3D Human Pose and Shape Estimation](https://mlsanity.com/paper/2011.14672/) (2020)
- [Resolving 3D Human Pose Ambiguities with 3D Scene Constraints](https://mlsanity.com/paper/1908.06963/) (2019)
- [ReMoDiffuse: Retrieval-Augmented Motion Diffusion Model](https://mlsanity.com/paper/2304.01116/) (2023)
- [Convolutional Mesh Regression for Single-Image Human Shape Reconstruction](https://mlsanity.com/paper/1905.03244/) (2019)

## Built on by (most cited first, 30 of 48)

- [GENMO: A GENeralist Model for Human MOtion](https://mlsanity.com/paper/2505.01425/) (2025)
- [HMD^2: Environment-aware Motion Generation from Single Egocentric Head-Mounted Device](https://mlsanity.com/paper/2409.13426/) (2024)
- [Scaling Large Motion Models with Million-Level Human Motions](https://mlsanity.com/paper/2410.03311/) (2024)
- [COIN: Control-Inpainting Diffusion Prior for Human and Camera Motion Estimation](https://mlsanity.com/paper/2408.16426/) (2024)
- [PRIMAL: Physically Reactive and Interactive Motor Model for Avatar Learning](https://mlsanity.com/paper/2503.17544/) (2025)
- [CoMotion: Concurrent Multi-person 3D Motion](https://mlsanity.com/paper/2504.12186/) (2025)
- [RealisMotion: Decomposed Human Motion Control and Video Generation in the World Space](https://mlsanity.com/paper/2508.08588/) (2025)
- [SceneMI: Motion In-betweening for Modeling Human-Scene Interactions](https://mlsanity.com/paper/2503.16289/) (2025)
- [Motion Generation: A Survey of Generative Approaches and Benchmarks](https://mlsanity.com/paper/2507.05419/) (2025)
- [EgoGrasp: World-Space Hand-Object Interaction Estimation from Egocentric Videos](https://mlsanity.com/paper/2601.01050/) (2026)
- [DuoMo: Dual Motion Diffusion for World-Space Human Reconstruction](https://mlsanity.com/paper/2603.03265/) (2026)
- [StableMotion: Training Motion Cleanup Models with Unpaired Corrupted Data](https://mlsanity.com/paper/2505.03154/) (2025)
- [Geometric Neural Distance Fields for Learning Human Motion Priors](https://mlsanity.com/paper/2509.09667/) (2025)
- [CHAMP: Conformalized 3D Human Multi-Hypothesis Pose Estimators](https://mlsanity.com/paper/2407.06141/) (2024)
- [Platypose: Calibrated Zero-Shot Multi-Hypothesis 3D Human Motion Estimation](https://mlsanity.com/paper/2403.06164/) (2024)
- [DyCrowd: Towards Dynamic Crowd Reconstruction from a Large-scene Video](https://mlsanity.com/paper/2508.12644/) (2025)
- [Group Inertial Poser: Multi-Person Pose and Global Translation from Sparse Inertial Sensors and Ultra-Wideband Ranging](https://mlsanity.com/paper/2510.21654/) (2025)
- [NymeriaPlus: Enriching Nymeria Dataset with Additional Annotations and Data](https://mlsanity.com/paper/2603.18496/) (2026)
- [CUPS: Improving Human Pose-Shape Estimators with Conformalized Deep Uncertainty](https://mlsanity.com/paper/2412.10431/) (2024)
- [SPG: Improving Motion Diffusion by Smooth Perturbation Guidance](https://mlsanity.com/paper/2503.02577/) (2025)
- [FLAM: Foundation Model-Based Body Stabilization for Humanoid Locomotion and Manipulation](https://mlsanity.com/paper/2503.22249/) (2025)
- [MeshMamba: State Space Models for Articulated 3D Mesh Generation and Reconstruction](https://mlsanity.com/paper/2507.15212/) (2025)
- [DiffCap: Diffusion-based Real-time Human Motion Capture using Sparse IMUs and a Monocular Camera](https://mlsanity.com/paper/2508.06139/) (2025)
- [Unconditional Human Motion and Shape Generation via Balanced Score-Based Diffusion](https://mlsanity.com/paper/2510.12537/) (2025)
- [Human Motion Prediction, Reconstruction, and Generation](https://mlsanity.com/paper/2502.15956/) (2025)
- [FUSION: Full-Body Unified Motion Prior for Body and Hands via Diffusion](https://mlsanity.com/paper/2601.03959/) (2026)
- [Masked Modeling for Human Motion Recovery Under Occlusions](https://mlsanity.com/paper/2601.16079/) (2026)
- [PDF-HR: Pose Distance Fields for Humanoid Robots](https://mlsanity.com/paper/2602.04851/) (2026)
- [MOCHI: Motion Enhancement of Collaborative Human-object Interactions](https://mlsanity.com/paper/2606.18243/) (2026)
- [EgoExoMoCap: Distributed Ego-Exo Human Motion Capture](https://mlsanity.com/paper/2607.15868/) (2026)

## Abstract (arXiv)

We propose RoHM, an approach for robust 3D human motion reconstruction from monocular RGB(-D) videos in the presence of noise and occlusions. Most previous approaches either train neural networks to directly regress motion in 3D or learn data-driven motion priors and combine them with optimization at test time. The former do not recover globally coherent motion and fail under occlusions; the latter are time-consuming, prone to local minima, and require manual tuning. To overcome these shortcomings, we exploit the iterative, denoising nature of diffusion models. RoHM is a novel diffusion-based motion model that, conditioned on noisy and occluded input data, reconstructs complete, plausible motions in consistent global coordinates. Given the complexity of the problem -- requiring one to address different tasks (denoising and infilling) in different solution spaces (local and global motion) -- we decompose it into two sub-tasks and learn two models, one for global trajectory and one for local motion. To capture the correlations between the two, we then introduce a novel conditioning module, combining it with an iterative inference scheme. We apply RoHM to a variety of tasks -- from motion reconstruction and denoising to spatial and temporal infilling. Extensive experiments on three popular datasets show that our method outperforms state-of-the-art approaches qualitatively and quantitatively, while being faster at test time. The code is available at https://sanweiliti.github.io/ROHM/ROHM.html.