# Learning to Estimate Robust 3D Human Mesh from In-the-Wild Crowded Scenes

arXiv: https://arxiv.org/abs/2104.07300 · HTML: https://mlsanity.com/paper/2104.07300/

## Key facts

- Cited by 65 papers in the map (data through 14 Oct 2026).
- Builds on 43 of its 59 references that are in the map.
- First posted on arXiv 15 Apr 2021.

## Builds on (most cited first, 30 of 43)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Expressive Body Capture: 3D Hands, Face, and Body from a Single Image](https://mlsanity.com/paper/1904.05866/) (2019)
- [Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1611.08050/) (2016)
- [End-to-end Recovery of Human Shape and Pose](https://mlsanity.com/paper/1712.06584/) (2017)
- [Learning to Reconstruct 3D Human Pose and Shape via Model-fitting in the Loop](https://mlsanity.com/paper/1909.12828/) (2019)
- [Simple Baselines for Human Pose Estimation and Tracking](https://mlsanity.com/paper/1804.06208/) (2018)
- [Monocular 3D Human Pose Estimation In The Wild Using Improved CNN Supervision](https://mlsanity.com/paper/1611.09813/) (2016)
- [A simple yet effective baseline for 3d human pose estimation](https://mlsanity.com/paper/1705.03098/) (2017)
- [Cascaded Pyramid Network for Multi-Person Pose Estimation](https://mlsanity.com/paper/1711.07319/) (2017)
- [I2L-MeshNet: Image-to-Lixel Prediction Network for Accurate 3D Human Pose and Mesh Estimation from a Single RGB Image](https://mlsanity.com/paper/2008.03713/) (2020)
- [Convolutional Mesh Regression for Single-Image Human Shape Reconstruction](https://mlsanity.com/paper/1905.03244/) (2019)
- [Pose2Mesh: Graph Convolutional Network for 3D Human Pose and Mesh Recovery from a 2D Human Pose](https://mlsanity.com/paper/2008.09047/) (2020)
- [HigherHRNet: Scale-Aware Representation Learning for Bottom-Up Human Pose Estimation](https://mlsanity.com/paper/1908.10357/) (2019)
- [Monocular, One-stage, Regression of Multiple 3D People](https://mlsanity.com/paper/2008.12272/) (2020)
- [CrowdPose: Efficient Crowded Scenes Pose Estimation and A New Benchmark](https://mlsanity.com/paper/1812.00324/) (2018)
- [Associative Embedding: End-to-End Learning for Joint Detection and Grouping](https://mlsanity.com/paper/1611.05424/) (2016)
- [Single-Shot Multi-Person 3D Pose Estimation From Monocular RGB](https://mlsanity.com/paper/1712.03453/) (2017)
- [Panoptic Studio: A Massively Multiview System for Social Interaction Capture](https://mlsanity.com/paper/1612.03153/) (2016)
- [PoseTrack: A Benchmark for Human Pose Estimation and Tracking](https://mlsanity.com/paper/1710.10000/) (2017)
- [Beyond Static Features for Temporally Consistent 3D Human Pose and Shape from a Video](https://mlsanity.com/paper/2011.08627/) (2020)
- [Coherent Reconstruction of Multiple Humans from a Single Image](https://mlsanity.com/paper/2006.08586/) (2020)
- [Camera Distance-aware Top-down Approach for 3D Multi-person Pose Estimation from a Single RGB Image](https://mlsanity.com/paper/1907.11346/) (2019)
- [Towards Accurate Multi-person Pose Estimation in the Wild](https://mlsanity.com/paper/1701.01779/) (2017)
- [Compositional Human Pose Regression](https://mlsanity.com/paper/1704.00159/) (2017)
- [DeepCut: Joint Subset Partition and Labeling for Multi Person Pose Estimation](https://mlsanity.com/paper/1511.06645/) (2015)
- [Pose2Seg: Detection Free Human Instance Segmentation](https://mlsanity.com/paper/1803.10683/) (2018)

## Built on by (most cited first, 30 of 65)

- [MotionBERT: A Unified Perspective on Learning Human Motion Representations](https://mlsanity.com/paper/2210.06551/) (2022)
- [PyMAF-X: Towards Well-aligned Full-body Model Regression from Monocular Images](https://mlsanity.com/paper/2207.06400/) (2022)
- [Putting People in their Place: Monocular Regression of 3D People in Depth](https://mlsanity.com/paper/2112.08274/) (2021)
- [Recovering 3D Human Mesh from Monocular Images: A Survey](https://mlsanity.com/paper/2203.01923/) (2022)
- [Multi-HMR: Multi-Person Whole-Body Human Mesh Recovery in a Single Shot](https://mlsanity.com/paper/2402.14654/) (2024)
- [HandOccNet: Occlusion-Robust 3D Hand Mesh Estimation Network](https://mlsanity.com/paper/2203.14564/) (2022)
- [TokenHMR: Advancing Human Mesh Recovery with a Tokenized Pose Representation](https://mlsanity.com/paper/2404.16752/) (2024)
- [3D Human Mesh Estimation from Virtual Markers](https://mlsanity.com/paper/2303.11726/) (2023)
- [Bringing Inputs to Shared Domains for 3D Interacting Hands Recovery in the Wild](https://mlsanity.com/paper/2303.13652/) (2023)
- [HAP: Structure-Aware Masked Image Modeling for Human-Centric Perception](https://mlsanity.com/paper/2310.20695/) (2023)
- [PSVT: End-to-End Multi-person 3D Pose and Shape Estimation with Progressive Video Transformers](https://mlsanity.com/paper/2303.09187/) (2023)
- [Joint Reconstruction of 3D Human and Object via Contact-Based Refinement Transformer](https://mlsanity.com/paper/2404.04819/) (2024)
- [Towards Open Domain Text-Driven Synthesis of Multi-Person Motions](https://mlsanity.com/paper/2405.18483/) (2024)
- [Distribution-Aligned Diffusion for Human Mesh Recovery](https://mlsanity.com/paper/2308.13369/) (2023)
- [Sampling is Matter: Point-guided 3D Human Mesh Reconstruction](https://mlsanity.com/paper/2304.09502/) (2023)
- [Promptable Game Models: Text-Guided Game Simulation via Masked Diffusion Models](https://mlsanity.com/paper/2303.13472/) (2023)
- [Hulk: A Universal Knowledge Translator for Human-Centric Tasks](https://mlsanity.com/paper/2312.01697/) (2023)
- [Closely Interactive Human Reconstruction with Proxemics and Physics-Guided Adaption](https://mlsanity.com/paper/2404.11291/) (2024)
- [PLIKS: A Pseudo-Linear Inverse Kinematic Solver for 3D Human Body Estimation](https://mlsanity.com/paper/2211.11734/) (2022)
- [Unsupervised Volumetric Animation](https://mlsanity.com/paper/2301.11326/) (2023)
- [Reconstructing Groups of People with Hypergraph Relational Reasoning](https://mlsanity.com/paper/2308.15844/) (2023)
- [DPMesh: Exploiting Diffusion Prior for Occluded Human Mesh Recovery](https://mlsanity.com/paper/2404.01424/) (2024)
- [SAT-HMR: Real-Time Multi-Person 3D Mesh Estimation via Scale-Adaptive Tokens](https://mlsanity.com/paper/2411.19824/) (2024)
- [BioPose: Biomechanically-accurate 3D Pose Estimation from Monocular Videos](https://mlsanity.com/paper/2501.07800/) (2025)
- [PoseBERT: A Generic Transformer Module for Temporal 3D Human Modeling](https://mlsanity.com/paper/2208.10211/) (2022)
- [MEGA: Masked Generative Autoencoder for Human Mesh Recovery](https://mlsanity.com/paper/2405.18839/) (2024)
- [Crowd3D: Towards Hundreds of People Reconstruction from a Single Image](https://mlsanity.com/paper/2301.09376/) (2023)
- [JOTR: 3D Joint Contrastive Learning with Transformers for Occluded Human Mesh Recovery](https://mlsanity.com/paper/2307.16377/) (2023)
- [Three Recipes for Better 3D Pseudo-GTs of 3D Human Mesh Estimation in the Wild](https://mlsanity.com/paper/2304.04875/) (2023)
- [3DPortraitGAN: Learning One-Quarter Headshot 3D GANs from a Single-View Portrait Dataset with Diverse Body Poses](https://mlsanity.com/paper/2307.14770/) (2023)

## Abstract (arXiv)

We consider the problem of recovering a single person's 3D human mesh from in-the-wild crowded scenes. While much progress has been in 3D human mesh estimation, existing methods struggle when test input has crowded scenes. The first reason for the failure is a domain gap between training and testing data. A motion capture dataset, which provides accurate 3D labels for training, lacks crowd data and impedes a network from learning crowded scene-robust image features of a target person. The second reason is a feature processing that spatially averages the feature map of a localized bounding box containing multiple people. Averaging the whole feature map makes a target person's feature indistinguishable from others. We present 3DCrowdNet that firstly explicitly targets in-the-wild crowded scenes and estimates a robust 3D human mesh by addressing the above issues. First, we leverage 2D human pose estimation that does not require a motion capture dataset with 3D labels for training and does not suffer from the domain gap. Second, we propose a joint-based regressor that distinguishes a target person's feature from others. Our joint-based regressor preserves the spatial activation of a target by sampling features from the target's joint locations and regresses human model parameters. As a result, 3DCrowdNet learns target-focused features and effectively excludes the irrelevant features of nearby persons. We conduct experiments on various benchmarks and prove the robustness of 3DCrowdNet to the in-the-wild crowded scenes both quantitatively and qualitatively. The code is available at https://github.com/hongsukchoi/3DCrowdNet_RELEASE.