Human Interaction-Aware 3D Reconstruction from a Single Image
Not on the map. Why?
Abstract (arXiv)
Reconstructing textured 3D human models from a single image is fundamental for AR/VR and digital human applications. However, existing methods mostly focus on single individuals and thus fail in multi-human scenes, where naive composition of individual reconstructions often leads to artifacts such as unrealistic overlaps, missing geometry in occluded regions, and distorted interactions. These limitations highlight the need for approaches that incorporate group-level context and interaction priors. We introduce a holistic method that explicitly models both group- and instance-level information. To mitigate perspective-induced geometric distortions, we first transform the input into a canonical orthographic space. Our primary component, Human Group-Instance Multi-View Diffusion (HUG-MVD), then generates complete multi-view normals and images by jointly modeling individuals and group context to resolve occlusions and proximity. Subsequently, the Human Group-Instance Geometric Reconstruction (HUG-GR) module optimizes the geometry by leveraging explicit, physics-based interaction priors to enforce physical plausibility and accurately model inter-human contact. Finally, the multi-view images are fused into a high-fidelity texture. Together, these components form our complete framework, HUG3D. Extensive experiments show that HUG3D significantly outperforms both single-human and existing multi-human methods, producing physically plausible, high-fidelity 3D reconstructions of interacting people from a single image. Project page: https://jongheean11.github.io/HUG3D_project
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 40 of its 51 references that are in the map. First posted on arXiv 7 Apr 2026.
Builds on
40 of 51 references.
References · 40
- 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
- 2023Segment Anythingcited by 8,158
- 2022Classifier-Free Diffusion Guidancecited by 5,717
- 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
- 2019Expressive Body Capture: 3D Hands, Face, and Body from a Single Imagecited by 1,600
- 2017End-to-end Recovery of Human Shape and Posecited by 883
- 2019PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitizationcited by 818
- 2020Accelerating 3D Deep Learning with PyTorch3Dcited by 781
- 2016Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Imagecited by 692
- 2019VIBE: Video Inference for Human Body Pose and Shape Estimationcited by 615
- 2020PIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitizationcited by 521
28 more
- 2024YOLOv11: An Overview of the Key Architectural Enhancementscited by 414
- 2021ICON: Implicit Clothed humans Obtained from Normalscited by 283
- 2021Function4D: Real-time Human Volumetric Capture from Very Sparse Consumer RGBD Sensorscited by 246
- 2018Total Capture: A 3D Deformation Model for Tracking Faces, Hands, and Bodiescited by 239
- 2022Towards Robust Blind Face Restoration with Codebook Lookup Transformercited by 230
- 2024Sapiens: Foundation for Human Vision Modelscited by 229
- 2018CrowdPose: Efficient Crowded Scenes Pose Estimation and A New Benchmarkcited by 214
- 2022ECON: Explicit Clothed humans Optimized via Normal integrationcited by 199
- 2020Populating 3D Scenes by Learning Human-Scene Interactioncited by 164
- 2021Putting People in their Place: Monocular Regression of 3D People in Depthcited by 141
- 2021Pixel Codec Avatarscited by 125
- 2024Era3D: High-Resolution Multiview Diffusion using Efficient Row-wise Attentioncited by 99
- 2024Multi-HMR: Multi-Person Whole-Body Human Mesh Recovery in a Single Shotcited by 98
- 2023Hi4D: 4D Instance Segmentation of Close Human Interactioncited by 87
- 2023SIFU: Side-view Conditioned Implicit Function for Real-world Usable Clothed Human Reconstructioncited by 77
- 2023SiTH: Single-view Textured Human Reconstruction with Image-Conditioned Diffusioncited by 72
- 2021DeepMultiCap: Performance Capture of Multiple Characters Using Sparse Multiview Camerascited by 67
- 2025LHM: Large Animatable Human Reconstruction Model from a Single Image in Secondscited by 61
- 2023Learning Locally Editable Virtual Humanscited by 54
- 2023Generative Proxemics: A Prior for 3D Social Interaction from Imagescited by 43
- 2024Instant3dit: Multiview Inpainting for Fast Editing of 3D Objectscited by 43
- 2024MVInpainter: Learning Multi-View Consistent Inpainting to Bridge 2D and 3D Editingcited by 30
- 2024Closely Interactive Human Reconstruction with Proxemics and Physics-Guided Adaptioncited by 18
- 2024Guess The Unseen: Dynamic 3D Scene Reconstruction from Partial 2D Glimpsescited by 17
- 2021Multi-person Implicit Reconstruction from a Single Imagecited by 13
- 2024MultiPly: Reconstruction of Multiple People from Monocular Video in the Wildcited by 12
- 2025MEAT: Multiview Diffusion Model for Human Generation on Megapixels with Mesh Attentioncited by 8
- 20243D Reconstruction of Interacting Multi-Person in Clothing from a Single Imagecited by 4
Built on by
Not cited yet.