mlsanityData to 14 Oct 2026

Learning Human Mesh Recovery in 3D Scenes

arXiv 2306.03847 ↗First posted 6 Jun 2023cs.CVcited by 13

Not on the map. Why?

Abstract (arXiv)

We present a novel method for recovering the absolute pose and shape of a human in a pre-scanned scene given a single image. Unlike previous methods that perform sceneaware mesh optimization, we propose to first estimate absolute position and dense scene contacts with a sparse 3D CNN, and later enhance a pretrained human mesh recovery network by cross-attention with the derived 3D scene cues. Joint learning on images and scene geometry enables our method to reduce the ambiguity caused by depth and occlusion, resulting in more reasonable global postures and contacts. Encoding scene-aware cues in the network also allows the proposed method to be optimization-free, and opens up the opportunity for real-time applications. The experiments show that the proposed network is capable of recovering accurate and physically-plausible meshes by a single forward pass and outperforms state-of-the-art methods in terms of both accuracy and speed.

Key facts. Cited by 13 papers in the map (data through 14 Oct 2026). Builds on 32 of its 41 references that are in the map. First posted on arXiv 6 Jun 2023.

Builds on

32 of 41 references.

References · 32

  1. 2017Attention Is All You Needcited by 60,361
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  3. 2020Transformers are RNNs: Fast Autoregressive Transformers with Linear Attentioncited by 2,022
  4. 2019Expressive Body Capture: 3D Hands, Face, and Body from a Single Imagecited by 1,612
  5. 2019SuperGlue: Learning Feature Matching with Graph Neural Networkscited by 1,443
  6. 2019Deep High-Resolution Representation Learning for Human Pose Estimationcited by 1,339
  7. 2017End-to-end Recovery of Human Shape and Posecited by 885
  8. 2016Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Imagecited by 695
  9. 2019Learning to Reconstruct 3D Human Pose and Shape via Model-fitting in the Loopcited by 627
  10. 2020Searching Efficient 3D Architectures with Sparse Point-Voxel Convolutioncited by 423
  11. 2020End-to-End Human Pose and Mesh Reconstruction with Transformerscited by 418
  12. 2021HuMoR: 3D Human Motion Model for Robust Pose Estimationcited by 324
20 more
  1. 2019Resolving 3D Human Pose Ambiguities with 3D Scene Constraintscited by 295
  2. 2020PREDATOR: Registration of 3D Point Clouds with Low Overlapcited by 290
  3. 2019Convolutional Mesh Regression for Single-Image Human Shape Reconstructioncited by 270
  4. 2021Mesh Graphormercited by 265
  5. 2022Embodied Hands: Modeling and Capturing Hands and Bodies Togethercited by 181
  6. 2022Capturing and Inferring Dense Full-Body Human-Scene Contactcited by 180
  7. 2020Populating 3D Scenes by Learning Human-Scene Interactioncited by 164
  8. 2020Perceiving 3D Human-Object Spatial Arrangements from a Single Image in the Wildcited by 152
  9. 2021SPEC: Seeing People in the Wild with an Estimated Cameracited by 135
  10. 2019Generating 3D People in Scenes without Peoplecited by 127
  11. 2020Exemplar Fine-Tuning for 3D Human Model Fitting Towards In-the-Wild 3D Human Pose Estimationcited by 113
  12. 2021Learning Motion Priors for 4D Human Body Capture in 3D Scenescited by 98
  13. 2022Pose-NDF: Modeling Human Pose Manifolds with Neural Distance Fieldscited by 85
  14. 2022TorchSparse: Efficient Point Cloud Inference Enginecited by 73
  15. 2020Holistic 3D Human and Scene Mesh Estimation from Single View Imagescited by 54
  16. 2020SMAP: Single-Shot Multi-Person Absolute 3D Pose Estimationcited by 52
  17. 2020PhysCap: Physically Plausible Monocular 3D Motion Capture in Real Timecited by 50
  18. 2022HULC: 3D Human Motion Capture with Pose Manifold Sampling and Dense Contact Guidancecited by 29
  19. 2022Embodied Scene-aware Human Pose Estimationcited by 28
  20. 2022MoCapDeform: Monocular 3D Human Motion Capture in Deformable Scenescited by 18

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 7; 2025: 3; 2026: 2.

13 papers.

Sort:

By area:

  1. 2023Dynamic Inertial Poser (DynaIP): Part-Based Motion Dynamics Learning for Enhanced Human Pose Estimation with Sparse Inertial Sensorscited by 30
  2. 2024Synergistic Global-space Camera and Human Reconstruction from Videoscited by 17
  3. 2024EnvPoser: Environment-aware Realistic Human Motion Estimation from Sparse Observations with Uncertainty Modelingcited by 11
  4. 2024UPose3D: Uncertainty-Aware 3D Human Pose Estimation with Cross-View and Temporal Cuescited by 6
  5. 2024Distribution and Depth-Aware Transformers for 3D Human Mesh Recoverycited by 5
  6. 2024Human-Aware 3D Scene Generation with Spatially-constrained Diffusion Modelscited by 5
  7. 2025mmDEAR: mmWave Point Cloud Density Enhancement for Accurate Human Body Reconstructioncited by 5
  8. 2024SkelFormer: Markerless 3D Pose and Shape Estimation using Skeletal Transformerscited by 2
  9. 2024Ask, Pose, Unite: Scaling Data Acquisition for Close Interactions with Vision Language Modelscited by 1
  10. 2025Mocap-2-to-3: Multi-view Lifting for Monocular Motion Recovery with 2D Pretrainingcited by 0
  11. 2025Avatar4D: Synthesizing Domain-Specific 4D Humans for Real-World Pose Estimationcited by 0
  12. 2026InterMesh: Explicit Interaction-Aware End-to-End Multi-Person Human Mesh Recoverycited by 0
1 more
  1. 2026Efficient Human-Contact Representation for Human-Scene Interactioncited by 0