mlsanityData to 7 Oct 2026

Human Interaction-Aware 3D Reconstruction from a Single Image

arXiv 2604.05436 ↗First posted 7 Apr 2026cs.CV cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Reconstructing textured 3D human models from a single image is fundamental for AR/VR and digital human applications. However, existing methods mostly focus on single individuals and thus fail in multi-human scenes, where naive composition of individual reconstructions often leads to artifacts such as unrealistic overlaps, missing geometry in occluded regions, and distorted interactions. These limitations highlight the need for approaches that incorporate group-level context and interaction priors. We introduce a holistic method that explicitly models both group- and instance-level information. To mitigate perspective-induced geometric distortions, we first transform the input into a canonical orthographic space. Our primary component, Human Group-Instance Multi-View Diffusion (HUG-MVD), then generates complete multi-view normals and images by jointly modeling individuals and group context to resolve occlusions and proximity. Subsequently, the Human Group-Instance Geometric Reconstruction (HUG-GR) module optimizes the geometry by leveraging explicit, physics-based interaction priors to enforce physical plausibility and accurately model inter-human contact. Finally, the multi-view images are fused into a high-fidelity texture. Together, these components form our complete framework, HUG3D. Extensive experiments show that HUG3D significantly outperforms both single-human and existing multi-human methods, producing physically plausible, high-fidelity 3D reconstructions of interacting people from a single image. Project page: https://jongheean11.github.io/HUG3D_project

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 40 of its 51 references that are in the map. First posted on arXiv 7 Apr 2026.

Builds on

40 of 51 references.

References · 40

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2023Segment Anythingcited by 8,158
  4. 2022Classifier-Free Diffusion Guidancecited by 5,717
  5. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  6. 2019Expressive Body Capture: 3D Hands, Face, and Body from a Single Imagecited by 1,600
  7. 2017End-to-end Recovery of Human Shape and Posecited by 883
  8. 2019PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitizationcited by 818
  9. 2020Accelerating 3D Deep Learning with PyTorch3Dcited by 781
  10. 2016Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Imagecited by 692
  11. 2019VIBE: Video Inference for Human Body Pose and Shape Estimationcited by 615
  12. 2020PIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitizationcited by 521
28 more
  1. 2024YOLOv11: An Overview of the Key Architectural Enhancementscited by 414
  2. 2021ICON: Implicit Clothed humans Obtained from Normalscited by 283
  3. 2021Function4D: Real-time Human Volumetric Capture from Very Sparse Consumer RGBD Sensorscited by 246
  4. 2018Total Capture: A 3D Deformation Model for Tracking Faces, Hands, and Bodiescited by 239
  5. 2022Towards Robust Blind Face Restoration with Codebook Lookup Transformercited by 230
  6. 2024Sapiens: Foundation for Human Vision Modelscited by 229
  7. 2018CrowdPose: Efficient Crowded Scenes Pose Estimation and A New Benchmarkcited by 214
  8. 2022ECON: Explicit Clothed humans Optimized via Normal integrationcited by 199
  9. 2020Populating 3D Scenes by Learning Human-Scene Interactioncited by 164
  10. 2021Putting People in their Place: Monocular Regression of 3D People in Depthcited by 141
  11. 2021Pixel Codec Avatarscited by 125
  12. 2024Era3D: High-Resolution Multiview Diffusion using Efficient Row-wise Attentioncited by 99
  13. 2024Multi-HMR: Multi-Person Whole-Body Human Mesh Recovery in a Single Shotcited by 98
  14. 2023Hi4D: 4D Instance Segmentation of Close Human Interactioncited by 87
  15. 2023SIFU: Side-view Conditioned Implicit Function for Real-world Usable Clothed Human Reconstructioncited by 77
  16. 2023SiTH: Single-view Textured Human Reconstruction with Image-Conditioned Diffusioncited by 72
  17. 2021DeepMultiCap: Performance Capture of Multiple Characters Using Sparse Multiview Camerascited by 67
  18. 2025LHM: Large Animatable Human Reconstruction Model from a Single Image in Secondscited by 61
  19. 2023Learning Locally Editable Virtual Humanscited by 54
  20. 2023Generative Proxemics: A Prior for 3D Social Interaction from Imagescited by 43
  21. 2024Instant3dit: Multiview Inpainting for Fast Editing of 3D Objectscited by 43
  22. 2024MVInpainter: Learning Multi-View Consistent Inpainting to Bridge 2D and 3D Editingcited by 30
  23. 2024Closely Interactive Human Reconstruction with Proxemics and Physics-Guided Adaptioncited by 18
  24. 2024Guess The Unseen: Dynamic 3D Scene Reconstruction from Partial 2D Glimpsescited by 17
  25. 2021Multi-person Implicit Reconstruction from a Single Imagecited by 13
  26. 2024MultiPly: Reconstruction of Multiple People from Monocular Video in the Wildcited by 12
  27. 2025MEAT: Multiview Diffusion Model for Human Generation on Megapixels with Mesh Attentioncited by 8
  28. 20243D Reconstruction of Interacting Multi-Person in Clothing from a Single Imagecited by 4

Built on by

Not cited yet.