# Score-Guided Diffusion for 3D Human Recovery

arXiv: https://arxiv.org/abs/2403.09623 · HTML: https://mlsanity.com/paper/2403.09623/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models.
- Builds on 58 of its 71 references that are in the map.
- First posted on arXiv 14 Mar 2024.

## Placed in

- Diffusion models / [3D, point cloud and scene generation](https://mlsanity.com/diffusion/3d-point-cloud-and-scene-generation/)

## Builds on (most cited first, 30 of 58)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [Expressive Body Capture: 3D Hands, Face, and Body from a Single Image](https://mlsanity.com/paper/1904.05866/) (2019)
- [Diffusion Posterior Sampling for General Noisy Inverse Problems](https://mlsanity.com/paper/2209.14687/) (2022)
- [On the Continuity of Rotation Representations in Neural Networks](https://mlsanity.com/paper/1812.07035/) (2018)
- [OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1812.08008/) (2018)
- [Human Motion Diffusion Model](https://mlsanity.com/paper/2209.14916/) (2022)
- [End-to-end Recovery of Human Shape and Pose](https://mlsanity.com/paper/1712.06584/) (2017)
- [Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Image](https://mlsanity.com/paper/1607.08128/) (2016)
- [Learning to Reconstruct 3D Human Pose and Shape via Model-fitting in the Loop](https://mlsanity.com/paper/1909.12828/) (2019)
- [VIBE: Video Inference for Human Body Pose and Shape Estimation](https://mlsanity.com/paper/1912.05656/) (2019)
- [Improving Diffusion Models for Inverse Problems using Manifold Constraints](https://mlsanity.com/paper/2206.00941/) (2022)
- [Monocular 3D Human Pose Estimation In The Wild Using Improved CNN Supervision](https://mlsanity.com/paper/1611.09813/) (2016)
- [End-to-End Human Pose and Mesh Reconstruction with Transformers](https://mlsanity.com/paper/2012.09760/) (2020)
- [PhysDiff: Physics-Guided Human Motion Diffusion Model](https://mlsanity.com/paper/2212.02500/) (2022)
- [Humans in 4D: Reconstructing and Tracking Humans with Transformers](https://mlsanity.com/paper/2305.20091/) (2023)
- [SVDiff: Compact Parameter Space for Diffusion Fine-Tuning](https://mlsanity.com/paper/2303.11305/) (2023)
- [PARE: Part Attention Regressor for 3D Human Body Estimation](https://mlsanity.com/paper/2104.08527/) (2021)
- [HuMoR: 3D Human Motion Model for Robust Pose Estimation](https://mlsanity.com/paper/2105.04668/) (2021)

## Built on by (most cited first, 30 of 39)

- [REWIND: Real-Time Egocentric Whole-Body Motion Diffusion with Exemplar-Based Identity Conditioning](https://mlsanity.com/paper/2504.04956/) (2025)
- [Single View Garment Reconstruction Using Diffusion Mapping Via Pattern Coordinates](https://mlsanity.com/paper/2504.08353/) (2025)
- [Diffusion Models in 3D Vision: A Survey](https://mlsanity.com/paper/2410.04738/) (2024)
- [ScoreHOI: Physically Plausible Reconstruction of Human-Object Interaction via Score-Guided Diffusion](https://mlsanity.com/paper/2509.07920/) (2025)
- [ODHSR: Online Dense 3D Reconstruction of Humans and Scenes from Monocular Videos](https://mlsanity.com/paper/2504.13167/) (2025)
- [ADHMR: Aligning Diffusion-based Human Mesh Recovery via Direct Preference Optimization](https://mlsanity.com/paper/2505.10250/) (2025)
- [DuoMo: Dual Motion Diffusion for World-Space Human Reconstruction](https://mlsanity.com/paper/2603.03265/) (2026)
- [DiSRT-In-Bed: Diffusion-Based Sim-to-Real Transfer Framework for In-Bed Human Mesh Recovery](https://mlsanity.com/paper/2504.03006/) (2025)
- [Multi-hypotheses Conditioned Point Cloud Diffusion for 3D Human Reconstruction from Occluded Images](https://mlsanity.com/paper/2409.18364/) (2024)
- [Reconstructing Close Human Interaction with Appearance and Proxemics Reasoning](https://mlsanity.com/paper/2507.02565/) (2025)
- [FastHMR: Accelerating Human Mesh Recovery via Token and Layer Merging with Diffusion Decoding](https://mlsanity.com/paper/2510.10868/) (2025)
- [Exploring the Potential of Synthetic Data to Replace Real Data](https://mlsanity.com/paper/2408.14559/) (2024)
- [Dance Style Recognition Using Laban Movement Analysis](https://mlsanity.com/paper/2504.21166/) (2025)
- [AJAHR: Amputated Joint Aware 3D Human Mesh Recovery](https://mlsanity.com/paper/2509.19939/) (2025)
- [PEAR: Pixel-aligned Expressive humAn mesh Recovery](https://mlsanity.com/paper/2601.22693/) (2026)
- [FMPose3D: monocular 3D pose estimation via flow matching](https://mlsanity.com/paper/2602.05755/) (2026)
- [DICE: Discrete Inversion Enabling Controllable Editing for Multinomial Diffusion and Masked Generative Models](https://mlsanity.com/paper/2410.08207/) (2024)
- [Multi-modal Pose Diffuser: A Multimodal Generative Conditional Pose Prior](https://mlsanity.com/paper/2410.14540/) (2024)
- [HeatFormer: A Neural Optimizer for Multiview Human Mesh Recovery](https://mlsanity.com/paper/2412.04456/) (2024)
- [CASIM: Composite Aware Semantic Injection for Text to Motion Generation](https://mlsanity.com/paper/2502.02063/) (2025)
- [SMPL-GPTexture: Dual-View 3D Human Texture Estimation using Text-to-Image Generation Models](https://mlsanity.com/paper/2504.13378/) (2025)
- [PHD: Personalized 3D Human Body Fitting with Point Diffusion](https://mlsanity.com/paper/2508.21257/) (2025)
- [Affordance-Guided Diffusion Prior for 3D Hand Reconstruction](https://mlsanity.com/paper/2510.00506/) (2025)
- [VLM-Guided Group Preference Alignment for Diffusion-based Human Mesh Recovery](https://mlsanity.com/paper/2602.19180/) (2026)
- [SynPlay: Large-Scale Synthetic Human Data with Real-World Diversity for Aerial-View Perception](https://mlsanity.com/paper/2408.11814/) (2024)
- [DiffProxy: Multi-View Human Mesh Recovery via Diffusion-Generated Dense Proxies](https://mlsanity.com/paper/2601.02267/) (2026)
- [AxisPose: Model-Free Matching-Free Single-Shot 6D Object Pose Estimation via Axis Generation](https://mlsanity.com/paper/2503.06660/) (2025)
- [SHaDe: Compact and Consistent Dynamic 3D Reconstruction via Tri-Plane Deformation and Latent Diffusion](https://mlsanity.com/paper/2505.16535/) (2025)
- [LieHMR: Autoregressive Human Mesh Recovery with SO(3) Diffusion](https://mlsanity.com/paper/2509.25739/) (2025)
- [Zero-shot Human Pose Estimation using Diffusion-based Inverse solvers](https://mlsanity.com/paper/2510.02043/) (2025)

## Abstract (arXiv)

We present Score-Guided Human Mesh Recovery (ScoreHMR), an approach for solving inverse problems for 3D human pose and shape reconstruction. These inverse problems involve fitting a human body model to image observations, traditionally solved through optimization techniques. ScoreHMR mimics model fitting approaches, but alignment with the image observation is achieved through score guidance in the latent space of a diffusion model. The diffusion model is trained to capture the conditional distribution of the human model parameters given an input image. By guiding its denoising process with a task-specific score, ScoreHMR effectively solves inverse problems for various applications without the need for retraining the task-agnostic diffusion model. We evaluate our approach on three settings/applications. These are: (i) single-frame model fitting; (ii) reconstruction from multiple uncalibrated views; (iii) reconstructing humans in video sequences. ScoreHMR consistently outperforms all optimization baselines on popular benchmarks across all settings. We make our code and models available at the https://statho.github.io/ScoreHMR.