Single-Shot Multi-Person 3D Pose Estimation From Monocular RGB
Not on the map. Why?
Abstract (arXiv)
We propose a new single-shot method for multi-person 3D pose estimation in general scenes from a monocular RGB camera. Our approach uses novel occlusion-robust pose-maps (ORPM) which enable full body pose inference even under strong partial occlusions by other people and objects in the scene. ORPM outputs a fixed number of maps which encode the 3D joint locations of all people in the scene. Body part associations allow us to infer 3D pose for an arbitrary number of people without explicit bounding box prediction. To train our approach we introduce MuCo-3DHP, the first large scale training data set showing real images of sophisticated multi-person interactions and occlusions. We synthesize a large corpus of multi-person images by compositing images of individual people (with ground truth from mutli-view performance capture). We evaluate our method on our new challenging 3D annotated multi-person test set MuPoTs-3D where we achieve state-of-the-art performance. To further stimulate research in multi-person 3D pose estimation, we will make our new datasets, and associated code publicly available for research purposes.
Key facts. Cited by 195 papers in the map (data through 14 Oct 2026). First posted on arXiv 9 Dec 2017.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 39; 2022: 47; 2023: 46; 2024: 28; 2025: 21; 2026: 14.
195 papers.
Sort:
By area:
- 2023Human Motion Diffusion as a Generative Priorcited by 357
- 2021Mesh Graphormercited by 265
- 2023BEDLAM: A Synthetic Dataset of Bodies Exhibiting Detailed Lifelike Animated Motioncited by 236
- 2021AGORA: Avatars in Geography Optimized for Regression Analysiscited by 205
- 2023SMPLer-X: Scaling Up Expressive Human Pose and Shape Estimationcited by 155
- 2021Putting People in their Place: Monocular Regression of 3D People in Depthcited by 142
- 2021SPEC: Seeing People in the Wild with an Estimated Cameracited by 135
- 2021SimPoE: Simulated Character Control for 3D Human Pose Estimationcited by 114
- 2022Recovering 3D Human Mesh from Monocular Images: A Surveycited by 103
- 2024Multi-HMR: Multi-Person Whole-Body Human Mesh Recovery in a Single Shotcited by 101
- 2021Learning Motion Priors for 4D Human Body Capture in 3D Scenescited by 98
- 2023TRACE: 5D Temporal Regression of Avatars with Dynamic Cameras in 3D Environmentscited by 91
38 more
- 2021PoseAug: A Differentiable Pose Augmentation Framework for 3D Human Pose Estimationcited by 89
- 2023Hi4D: 4D Instance Segmentation of Close Human Interactioncited by 87
- 2022Cross-Attention of Disentangled Modalities for 3D Human Mesh Recovery with Transformerscited by 81
- 2021Multi-Person Extreme Motion Predictioncited by 75
- 2022LASSIE: Learning Articulated Shapes from Sparse Image Ensemble via 3D Part Discoverycited by 66
- 2021Learning to Estimate Robust 3D Human Mesh from In-the-Wild Crowded Scenescited by 65
- 2021Playing for 3D Human Recoverycited by 64
- 2021SmoothNet: A Plug-and-Play Network for Refining Human Poses in Videoscited by 64
- 2021Multi-Person 3D Motion Prediction with Multi-Range Transformerscited by 63
- 2023EgoHumans: An Egocentric 3D Multi-Human Benchmarkcited by 60
- 2024Neural Localizer Fields for Continuous 3D Human Pose and Shape Estimationcited by 59
- 2022UnrealEgo: A New Dataset for Robust Egocentric 3D Human Motion Capturecited by 57
- 2022MIME: Human-Aware 3D Scene Generationcited by 57
- 2024HOI-M3:Capture Multiple Humans and Objects Interaction within Contextual Environmentcited by 53
- 2021Recent Advances in Monocular 2D and 3D Human Pose Estimation: A Deep Learning Perspectivecited by 50
- 2021THUNDR: Transformer-based 3D HUmaN Reconstruction with Markerscited by 50
- 2022Learning 3D Human Pose Estimation from Dozens of Datasets using a Geometry-Aware Autoencoder to Bridge Between Skeleton Formatscited by 50
- 2023HybrIK-X: Hybrid Analytical-Neural Inverse Kinematics for Whole-body Mesh Recoverycited by 49
- 2021Reconstructing 3D Human Pose by Watching Humans in the Mirrorcited by 47
- 2021Body Meshes as Pointscited by 45
- 2022D&D: Learning Human Dynamics from Dynamic Cameracited by 41
- 2022TORE: Token Reduction for Efficient Human Mesh Recovery with Transformercited by 41
- 2023Zolly: Zoom Focal Length Correctly for Perspective-Distorted Human Mesh Reconstructioncited by 41
- 2022Differentiable Dynamics for Articulated 3d Human Motion Reconstructioncited by 39
- 2021Graph-Based 3D Multi-Person Pose Estimation Using Multi-View Imagescited by 38
- 2021Lightweight Multi-person Total Motion Capture Using Sparse Multi-view Camerascited by 33
- 2021AdaptPose: Cross-Dataset Adaptation for 3D Human Pose Estimation by Learnable Motion Generationcited by 32
- 2023PSVT: End-to-End Multi-person 3D Pose and Shape Estimation with Progressive Video Transformerscited by 32
- 2025SMPLest-X: Ultimate Scaling for Expressive Human Pose and Shape Estimationcited by 32
- 2025InterAct: Advancing Large-Scale Versatile 3D Human-Object Interaction Generationcited by 32
- 2024Harmony4D: A Video Dataset for In-The-Wild Close Human Interactionscited by 31
- 2024Multi-agent Long-term 3D Human Pose Forecasting via Interaction-aware Trajectory Conditioningcited by 30
- 2021HSPACE: Synthetic Parametric Humans Animated in Complex Environmentscited by 29
- 2024Towards Open Domain Text-Driven Synthesis of Multi-Person Motionscited by 29
- 2022Benchmarking and Analyzing 3D Human Pose and Shape Estimation Beyond Algorithmscited by 28
- 2023Stochastic Multi-Person 3D Motion Forecastingcited by 28
- 20233D-Aware Neural Body Fitting for Occlusion Robust 3D Human Pose Estimationcited by 28
- 2023Distribution-Aligned Diffusion for Human Mesh Recoverycited by 27
50 most cited of 195 All 195 (JSON)
- 2026Residual Flow Matching with Dynamic Cross-Interaction for 3D Multi-Person Motion Prediction4 Aug 2026
- 2026DETRAM: End-to-end DEtection, Tracking and Recovery of HumAn Meshes10 Jul 2026
- 2026Multi-THuMBS: Multi-person Tracking of 3D Human Meshes Beyond Video Shots2 Jul 2026
- 2026DanceDuo: Bridging Human Movement and AI Choreography25 Jun 2026
- 2026Where Will They Go? Modelling Multimodal Pedestrian Manoeuvres from Ego-centric Videos17 Jun 2026
- 2026MoPO: Incorporating Motion Prior for Occluded Human Mesh Recovery11 May 2026
- 2026InterMesh: Explicit Interaction-Aware End-to-End Multi-Person Human Mesh Recovery6 May 2026
- 20262D Pre-Training for 3D Pose Estimation19 Apr 2026
- 2026MuPPet: Multi-person 2D-to-3D Pose Lifting8 Apr 2026
- 2026Learning Quantised Structure-Preserving Motion Representations for Dance Fingerprinting1 Apr 2026
- 2026PoseDreamer: Scalable and Photorealistic Human Data Generation Pipeline with Diffusion Models30 Mar 2026
- 2026Unleashing Guidance Without Classifiers for Human-Object Interaction Animation26 Mar 2026
38 more
- 2026Modeling 3D Pedestrian-Vehicle Interactions for Vehicle-Conditioned Pose Forecasting9 Feb 2026
- 2026JRDB-Pose3D: A Multi-person 3D Human Pose and Shape Estimation Dataset for Robotics3 Feb 2026
- 2025Spatiotemporal-Untrammelled Mixture of Experts for Multi-Person Motion Prediction25 Dec 2025
- 2025Towards Metric-Aware Multi-Person Mesh Recovery by Jointly Optimizing Human Crowd in Camera Space17 Nov 2025
- 2025SasMamba: A Lightweight Structure-Aware Stride State Space Model for 3D Human Pose Estimation12 Nov 2025
- 2025From Static to Dynamic: a Survey of Topology-Aware Perception in Autonomous Driving28 Sep 2025
- 2025InterAct: Advancing Large-Scale Versatile 3D Human-Object Interaction Generation11 Sep 2025
- 2025Waymo-3DSkelMo: A Multi-Agent 3D Skeletal Motion Dataset for Pedestrian Interaction Modeling in Autonomous Driving13 Aug 2025
- 2025KASportsFormer: Kinematic Anatomy Enhanced Transformer for 3D Human Pose Estimation on Short Sports Scene Video28 Jul 2025
- 2025CoMPAS3D: A Dataset and Benchmark for Interactive Motion25 Jul 2025
- 2025Physics-based Human Pose Estimation from a Single Moving RGB Camera23 Jul 2025
- 2025Efficient Multi-Person Motion Prediction by Lightweight Spatial and Temporal Interactions13 Jul 2025
- 2025Systematic Comparison of Projection Methods for Monocular 3D Human Pose Estimation on Fisheye Images24 Jun 2025
- 2025MAMMA: Markerless & Automatic Multi-Person Motion Action Capture16 Jun 2025
- 2025Multi-Person Interaction Generation from Two-Person Motion Priors23 May 2025
- 2025SocialGen: Modeling Multi-Human Social Interaction with Language Models28 Mar 2025
- 2025PoseSyn: Synthesizing Diverse 3D Pose Data from In-the-Wild 2D Data17 Mar 2025
- 2025A Survey on Human Interaction Motion Generation17 Mar 2025
- 2025EventEgo3D++: 3D Human Motion Capture from a Head-Mounted Event Camera11 Feb 2025
- 2025BlanketGen2-Fit3D: Synthetic Blanket Augmentation Towards Improving Real-World In-Bed Blanket Occluded Human Pose Estimation21 Jan 2025
- 2025SMPLest-X: Ultimate Scaling for Expressive Human Pose and Shape Estimation16 Jan 2025
- 2025Leveraging 2D Masked Reconstruction for Domain Adaptation of 3D Pose Estimation14 Jan 2025
- 2025WorldPose: A World Cup Dataset for Global 3D Human Pose Estimation6 Jan 2025
- 2024CondiMen: Conditional Multi-Person Mesh Recovery17 Dec 2024
- 2024SAT-HMR: Real-Time Multi-Person 3D Mesh Estimation via Scale-Adaptive Tokens29 Nov 2024
- 2024RCR: Robust Crowd Reconstruction with Upright Space from a Single Large-scene Image9 Nov 2024
- 2024UnityGraph: Unified Learning of Spatio-temporal features for Multi-person Motion Prediction6 Nov 2024
- 2024Relation Learning and Aggregate-attention for Multi-person Motion Prediction6 Nov 2024
- 2024Harmony4D: A Video Dataset for In-The-Wild Close Human Interactions27 Oct 2024
- 2024Ask, Pose, Unite: Scaling Data Acquisition for Close Interactions with Vision Language Models1 Oct 2024
- 2024Dense Hand-Object(HO) GraspNet with Full Grasping Taxonomy and Dynamics6 Sep 2024
- 2024Multi-view Pose Fusion for Occlusion-Aware 3D Human Pose Estimation28 Aug 2024
- 2024Motion Capture from Inertial and Vision Sensors23 Jul 2024
- 2024Neural Localizer Fields for Continuous 3D Human Pose and Shape Estimation10 Jul 2024
- 2024Human Modelling and Pose Estimation Overview27 Jun 2024
- 20243D WholeBody Pose Estimation based on Semantic Graph Attention Network and Distance Information3 Jun 2024
- 2024Towards Open Domain Text-Driven Synthesis of Multi-Person Motions28 May 2024
- 2024Synergistic Global-space Camera and Human Reconstruction from Videos23 May 2024
50 newest of 195 All 195 (JSON)
- 2023Human Motion Diffusion as a Generative Priorcited by 357
- 2023InterControl: Zero-shot Human Interaction Generation by Controlling Every Jointcited by 23
- 2024DPMesh: Exploiting Diffusion Prior for Occluded Human Mesh Recoverycited by 17
- 2025Multi-Person Interaction Generation from Two-Person Motion Priorscited by 8
- 2024ContactGen: Contact-Guided Interactive 3D Human Generation for Partnerscited by 3
- 2026PoseDreamer: Scalable and Photorealistic Human Data Generation Pipeline with Diffusion Modelscited by 0
- 2026Residual Flow Matching with Dynamic Cross-Interaction for 3D Multi-Person Motion Predictioncited by 0
- 2026Residual Flow Matching with Dynamic Cross-Interaction for 3D Multi-Person Motion Prediction4 Aug 2026
- 2026PoseDreamer: Scalable and Photorealistic Human Data Generation Pipeline with Diffusion Models30 Mar 2026
- 2025Multi-Person Interaction Generation from Two-Person Motion Priors23 May 2025
- 2024DPMesh: Exploiting Diffusion Prior for Occluded Human Mesh Recovery1 Apr 2024
- 2024ContactGen: Contact-Guided Interactive 3D Human Generation for Partners30 Jan 2024
- 2023InterControl: Zero-shot Human Interaction Generation by Controlling Every Joint27 Nov 2023
- 2023Human Motion Diffusion as a Generative Prior2 Mar 2023
- 2023Social Motion Prediction with Cognitive Hierarchiescited by 9
- 2023Social Motion Prediction with Cognitive Hierarchies8 Nov 2023
- 2021Mesh Graphormercited by 265
- 2023BEDLAM: A Synthetic Dataset of Bodies Exhibiting Detailed Lifelike Animated Motioncited by 236
- 2021AGORA: Avatars in Geography Optimized for Regression Analysiscited by 205
- 2023SMPLer-X: Scaling Up Expressive Human Pose and Shape Estimationcited by 155
- 2021Putting People in their Place: Monocular Regression of 3D People in Depthcited by 142
- 2021SPEC: Seeing People in the Wild with an Estimated Cameracited by 135
- 2021SimPoE: Simulated Character Control for 3D Human Pose Estimationcited by 114
- 2022Recovering 3D Human Mesh from Monocular Images: A Surveycited by 103
- 2024Multi-HMR: Multi-Person Whole-Body Human Mesh Recovery in a Single Shotcited by 101
- 2021Learning Motion Priors for 4D Human Body Capture in 3D Scenescited by 98
- 2023TRACE: 5D Temporal Regression of Avatars with Dynamic Cameras in 3D Environmentscited by 91
- 2021PoseAug: A Differentiable Pose Augmentation Framework for 3D Human Pose Estimationcited by 89
12 most cited of 183 All 183 (JSON)
- 2026DETRAM: End-to-end DEtection, Tracking and Recovery of HumAn Meshes10 Jul 2026
- 2026Multi-THuMBS: Multi-person Tracking of 3D Human Meshes Beyond Video Shots2 Jul 2026
- 2026DanceDuo: Bridging Human Movement and AI Choreography25 Jun 2026
- 2026Where Will They Go? Modelling Multimodal Pedestrian Manoeuvres from Ego-centric Videos17 Jun 2026
- 2026MoPO: Incorporating Motion Prior for Occluded Human Mesh Recovery11 May 2026
- 2026InterMesh: Explicit Interaction-Aware End-to-End Multi-Person Human Mesh Recovery6 May 2026
- 20262D Pre-Training for 3D Pose Estimation19 Apr 2026
- 2026MuPPet: Multi-person 2D-to-3D Pose Lifting8 Apr 2026
- 2026Learning Quantised Structure-Preserving Motion Representations for Dance Fingerprinting1 Apr 2026
- 2026Unleashing Guidance Without Classifiers for Human-Object Interaction Animation26 Mar 2026
- 2026Modeling 3D Pedestrian-Vehicle Interactions for Vehicle-Conditioned Pose Forecasting9 Feb 2026
- 2026JRDB-Pose3D: A Multi-person 3D Human Pose and Shape Estimation Dataset for Robotics3 Feb 2026
12 newest of 183 All 183 (JSON)