mlsanityData to 14 Oct 2026

Probabilistic Human Mesh Recovery in 3D Scenes from Egocentric Views

arXiv 2304.06024 ↗First posted 12 Apr 2023cs.CV cs.AIcited by 33

Not on the map. Why?

Abstract (arXiv)

Automatic perception of human behaviors during social interactions is crucial for AR/VR applications, and an essential component is estimation of plausible 3D human pose and shape of our social partners from the egocentric view. One of the biggest challenges of this task is severe body truncation due to close social distances in egocentric scenarios, which brings large pose ambiguities for unseen body parts. To tackle this challenge, we propose a novel scene-conditioned diffusion method to model the body pose distribution. Conditioned on the 3D scene geometry, the diffusion model generates bodies in plausible human-scene interactions, with the sampling guided by a physics-based collision score to further resolve human-scene inter-penetrations. The classifier-free training enables flexible sampling with different conditions and enhanced diversity. A visibility-aware graph convolution model guided by per-joint visibility serves as the diffusion denoiser to incorporate inter-joint dependencies and per-body-part control. Extensive evaluations show that our method generates bodies in plausible interactions with 3D scenes, achieving both superior accuracy for visible joints and diversity for invisible body parts. The code is available at https://sanweiliti.github.io/egohmr/egohmr.html.

Key facts. Cited by 33 papers in the map (data through 14 Oct 2026). Builds on 71 of its 85 references that are in the map. First posted on arXiv 12 Apr 2023.

Builds on

71 of 85 references.

References · 71

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
  4. 2020Denoising Diffusion Implicit Modelscited by 9,781
  5. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  6. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  7. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  8. 2022Classifier-Free Diffusion Guidancecited by 5,847
  9. 2019Expressive Body Capture: 3D Hands, Face, and Body from a Single Imagecited by 1,612
  10. 2018On the Continuity of Rotation Representations in Neural Networkscited by 1,214
  11. 2018OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,162
  12. 2022Human Motion Diffusion Modelcited by 1,080
38 more
  1. 2017End-to-end Recovery of Human Shape and Posecited by 885
  2. 2022MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Modelcited by 780
  3. 2016Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Imagecited by 695
  4. 2019Learning to Reconstruct 3D Human Pose and Shape via Model-fitting in the Loopcited by 627
  5. 2022Executing your Commands via Motion Diffusion in Latent Spacecited by 575
  6. 2016Monocular 3D Human Pose Estimation In The Wild Using Improved CNN Supervisioncited by 494
  7. 2020End-to-End Human Pose and Mesh Reconstruction with Transformerscited by 418
  8. 2020GRAB: A Dataset of Whole-Body Human Grasping of Objectscited by 405
  9. 2022PhysDiff: Physics-Guided Human Motion Diffusion Modelcited by 368
  10. 2021PARE: Part Attention Regressor for 3D Human Body Estimationcited by 337
  11. 2020HybrIK: A Hybrid Analytical-Neural Inverse Kinematics Solution for 3D Human Pose and Shape Estimationcited by 307
  12. 2019Resolving 3D Human Pose Ambiguities with 3D Scene Constraintscited by 295
  13. 2022EDGE: Editable Dance Generation From Musiccited by 287
  14. 2020I2L-MeshNet: Image-to-Lixel Prediction Network for Accurate 3D Human Pose and Mesh Estimation from a Single RGB Imagecited by 274
  15. 2019Convolutional Mesh Regression for Single-Image Human Shape Reconstructioncited by 270
  16. 2020Pose2Mesh: Graph Convolutional Network for 3D Human Pose and Mesh Recovery from a 2D Human Posecited by 248
  17. 2023Diffusion-based Generation, Optimization, and Planning in 3D Scenescited by 239
  18. 2022CLIFF: Carrying Location Information in Full Frames into Human Pose and Shape Estimationcited by 238
  19. 2022FLAME: Free-form Language-based Motion Synthesis & Editingcited by 217
  20. 2017Unite the People: Closing the Loop Between 3D and 2D Human Representationscited by 205
  21. 2021Stochastic Scene-Aware Motion Predictioncited by 203
  22. 2022Listen, Denoise, Action! Audio-Driven Motion Synthesis with Diffusion Modelscited by 190
  23. 2018Neural Body Fitting: Unifying Deep Learning and Model-Based Human Pose and Shape Estimationcited by 185
  24. 2021Probabilistic Modeling for Human Mesh Recoverycited by 153
  25. 2023Decoupling Human and Camera Motion from Videos in the Wildcited by 142
  26. 2021EgoBody: Human Body Shape and Motion of Interacting People from Head-Mounted Devicescited by 139
  27. 2021Human POSEitioning System (HPS): 3D Human Pose Estimation and Self-localization in Large Scenes from Body-Mounted Sensorscited by 137
  28. 2021SPEC: Seeing People in the Wild with an Estimated Cameracited by 135
  29. 2019Generating 3D People in Scenes without Peoplecited by 127
  30. 2021LEAP: Learning Articulated Occupancy of Peoplecited by 107
  31. 2019DenseRaC: Joint 3D Pose and Shape Estimation by Dense Render-and-Comparecited by 100
  32. 2022LaMAR: Benchmarking Localization and Mapping for Augmented Realitycited by 100
  33. 2021Learning Motion Priors for 4D Human Body Capture in 3D Scenescited by 98
  34. 2019Generating Multiple Hypotheses for 3D Human Pose Estimation with Mixture Density Networkcited by 92
  35. 2021On Self-Contact and Human Posecited by 89
  36. 2019Ego-Pose Estimation and Forecasting as Real-Time PD Controlcited by 87
  37. 2021Probabilistic Monocular 3D Human Pose Estimation with Normalizing Flowscited by 85
  38. 2019You2Me: Inferring Body Pose in Egocentric Video via First and Second Person Interactionscited by 82

50 most cited of 71 All 71 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 19; 2025: 7; 2026: 5.

33 papers.

Sort:

By area:

  1. 2024TRAM: Global Trajectory and Motion of 3D Humans from in-the-wild Videoscited by 106
  2. 2024Estimating Body and Hand Motion in an Ego-sensed Worldcited by 58
  3. 2024RoHM: Robust Human Motion Reconstruction via Diffusioncited by 48
  4. 2023Egocentric Whole-Body Motion Capture with FisheyeViT and Diffusion-Based Motion Refinementcited by 31
  5. 2024EventEgo3D: 3D Human Motion Capture from Egocentric Event Streamscited by 29
  6. 2024EgoGen: An Egocentric Synthetic Data Generatorcited by 27
  7. 2024HMD^2: Environment-aware Motion Generation from Single Egocentric Head-Mounted Devicecited by 27
  8. 2024EgoChoir: Capturing 3D Human-Object Interaction Regions from Egocentric Viewscited by 26
  9. 20233D Human Pose Perception from Egocentric Stereo Videoscited by 23
  10. 2024Diff-IP2D: Diffusion-Based Hand-Object Interaction Prediction on Egocentric Videoscited by 21
  11. 2024Lodge++: High-quality and Long Dance Generation with Vivid Choreography Patternscited by 21
  12. 2024MADiff: Motion-Aware Mamba Diffusion Models for Hand Trajectory Prediction on Egocentric Videoscited by 18
21 more
  1. 2025The Invisible EgoHand: 3D Hand Forecasting through EgoBody Pose Estimationcited by 16
  2. 2025REWIND: Real-Time Egocentric Whole-Body Motion Diffusion with Exemplar-Based Identity Conditioningcited by 14
  3. 2024EgoCast: Forecasting Egocentric Human Pose in the Wildcited by 11
  4. 2024EnvPoser: Environment-aware Realistic Human Motion Estimation from Sparse Observations with Uncertainty Modelingcited by 11
  5. 2025VolumetricSMPL: A Neural Volumetric Body Model for Efficient Interactions, Contacts, and Collisionscited by 9
  6. 2024A Survey on 3D Egocentric Human Pose Estimationcited by 8
  7. 2025Bring Your Rear Cameras for Egocentric 3D Human Pose Estimationcited by 7
  8. 2025ADHMR: Aligning Diffusion-based Human Mesh Recovery via Direct Preference Optimizationcited by 7
  9. 2024Multi-hypotheses Conditioned Point Cloud Diffusion for 3D Human Reconstruction from Occluded Imagescited by 5
  10. 2024CondiMen: Conditional Multi-Person Mesh Recoverycited by 5
  11. 2024Utilizing Uncertainty in 2D Pose Detectors for Probabilistic 3D Human Mesh Recoverycited by 4
  12. 2024Platypose: Calibrated Zero-Shot Multi-Hypothesis 3D Human Motion Estimationcited by 3
  13. 2026Masked Modeling for Human Motion Recovery Under Occlusionscited by 1
  14. 2026EgoExoMoCap: Distributed Ego-Exo Human Motion Capturecited by 1
  15. 2024Multi-RoI Human Mesh Recovery with Camera Consistency and Contrastive Lossescited by 0
  16. 2024Social EgoMesh Estimationcited by 0
  17. 2025LieHMR: Autoregressive Human Mesh Recovery with SO(3) Diffusioncited by 0
  18. 2025On the Use of Hierarchical Vision Foundation Models for Low-Cost Human Mesh Recovery and Pose Estimationcited by 0
  19. 2026RoSHI: A Versatile Robot-oriented Suit for Human Data In-the-Wildcited by 0
  20. 2026InterMesh: Explicit Interaction-Aware End-to-End Multi-Person Human Mesh Recoverycited by 0
  21. 2026HumanFlow -- Diffusion-Driven MAV Navigation Among Humans via Tightly-Coupled Motion Tracking, Forecasting, and Controlcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Synthesizing Moving People with 3D Control
  2. 2025PHD: Personalized 3D Human Body Fitting with Point Diffusion
  3. 2023DiffHPE: Robust, Coherent 3D Human Pose Lifting with Diffusion
  4. 2026EgoPoseFormer v2: Accurate Egocentric Human Motion Estimation for AR/VR
  5. 2025ScoreHOI: Physically Plausible Reconstruction of Human-Object Interaction via Score-Guided Diffusion
  6. 2023Template Free Reconstruction of Human-object Interaction with Procedural Interaction Generation
  7. 2023PrimDiffusion: Volumetric Primitives Diffusion for 3D Human Generation
  8. 2024Diffusion Models are Efficient Data Generators for Human Mesh Recovery