# 3D Reconstruction of Interacting Multi-Person in Clothing from a Single Image

arXiv: https://arxiv.org/abs/2401.06415 · HTML: https://mlsanity.com/paper/2401.06415/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 44 of its 56 references that are in the map.
- First posted on arXiv 12 Jan 2024.

## Builds on (most cited first, 30 of 44)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [End-to-end Recovery of Human Shape and Pose](https://mlsanity.com/paper/1712.06584/) (2017)
- [PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization](https://mlsanity.com/paper/1905.05172/) (2019)
- [Learning to Reconstruct 3D Human Pose and Shape via Model-fitting in the Loop](https://mlsanity.com/paper/1909.12828/) (2019)
- [PIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitization](https://mlsanity.com/paper/2004.00452/) (2020)
- [PARE: Part Attention Regressor for 3D Human Body Estimation](https://mlsanity.com/paper/2104.08527/) (2021)
- [HybrIK: A Hybrid Analytical-Neural Inverse Kinematics Solution for 3D Human Pose and Shape Estimation](https://mlsanity.com/paper/2011.14672/) (2020)
- [Resolving 3D Human Pose Ambiguities with 3D Scene Constraints](https://mlsanity.com/paper/1908.06963/) (2019)
- [ICON: Implicit Clothed humans Obtained from Normals](https://mlsanity.com/paper/2112.09127/) (2021)
- [ARCH: Animatable Reconstruction of Clothed Humans](https://mlsanity.com/paper/2004.04572/) (2020)
- [BEHAVE: Dataset and Method for Tracking Human Object Interactions](https://mlsanity.com/paper/2204.06950/) (2022)
- [Function4D: Real-time Human Volumetric Capture from Very Sparse Consumer RGBD Sensors](https://mlsanity.com/paper/2105.01859/) (2021)
- [PaMIR: Parametric Model-Conditioned Implicit Representation for Image-based Human Reconstruction](https://mlsanity.com/paper/2007.03858/) (2020)
- [SNARF: Differentiable Forward Skinning for Animating Non-Rigid Neural Implicit Shapes](https://mlsanity.com/paper/2104.03953/) (2021)
- [Monocular, One-stage, Regression of Multiple 3D People](https://mlsanity.com/paper/2008.12272/) (2020)
- [ECON: Explicit Clothed humans Optimized via Normal integration](https://mlsanity.com/paper/2212.07422/) (2022)
- [DeepHuman: 3D Human Reconstruction from a Single Image](https://mlsanity.com/paper/1903.06473/) (2019)
- [SCANimate: Weakly Supervised Learning of Skinned Clothed Avatar Networks](https://mlsanity.com/paper/2104.03313/) (2021)
- [Self-Correction for Human Parsing](https://mlsanity.com/paper/1910.09777/) (2019)
- [Capturing and Inferring Dense Full-Body Human-Scene Contact](https://mlsanity.com/paper/2206.09553/) (2022)
- [ARCH++: Animation-Ready Clothed Human Reconstruction Revisited](https://mlsanity.com/paper/2108.07845/) (2021)
- [Learning to Reconstruct People in Clothing from a Single RGB Camera](https://mlsanity.com/paper/1903.05885/) (2019)
- [Tex2Shape: Detailed Full Human Body Geometry From a Single Image](https://mlsanity.com/paper/1904.08645/) (2019)
- [Putting People in their Place: Monocular Regression of 3D People in Depth](https://mlsanity.com/paper/2112.08274/) (2021)
- [Coherent Reconstruction of Multiple Humans from a Single Image](https://mlsanity.com/paper/2006.08586/) (2020)
- [Photorealistic Monocular 3D Reconstruction of Humans Wearing Clothing](https://mlsanity.com/paper/2204.08906/) (2022)
- [Geo-PIFu: Geometry and Pixel Aligned Implicit Functions for Single-view Human Reconstruction](https://mlsanity.com/paper/2006.08072/) (2020)
- [Neural-GIF: Neural Generalized Implicit Functions for Animating People in Clothing](https://mlsanity.com/paper/2108.08807/) (2021)
- [CHORE: Contact, Human and Object REconstruction from a single RGB image](https://mlsanity.com/paper/2204.02445/) (2022)

## Built on by (most cited first, 4 of 4)

- [MultiPly: Reconstruction of Multiple People from Monocular Video in the Wild](https://mlsanity.com/paper/2406.01595/) (2024)
- [A Comparative Study of Garment Draping Techniques](https://mlsanity.com/paper/2405.11056/) (2024)
- [Ask, Pose, Unite: Scaling Data Acquisition for Close Interactions with Vision Language Models](https://mlsanity.com/paper/2410.00309/) (2024)
- [Human Interaction-Aware 3D Reconstruction from a Single Image](https://mlsanity.com/paper/2604.05436/) (2026)

## Abstract (arXiv)

This paper introduces a novel pipeline to reconstruct the geometry of interacting multi-person in clothing on a globally coherent scene space from a single image. The main challenge arises from the occlusion: a part of a human body is not visible from a single view due to the occlusion by others or the self, which introduces missing geometry and physical implausibility (e.g., penetration). We overcome this challenge by utilizing two human priors for complete 3D geometry and surface contacts. For the geometry prior, an encoder learns to regress the image of a person with missing body parts to the latent vectors; a decoder decodes these vectors to produce 3D features of the associated geometry; and an implicit network combines these features with a surface normal map to reconstruct a complete and detailed 3D humans. For the contact prior, we develop an image-space contact detector that outputs a probability distribution of surface contacts between people in 3D. We use these priors to globally refine the body poses, enabling the penetration-free and accurate reconstruction of interacting multi-person in clothing on the scene space. The results demonstrate that our method is complete, globally coherent, and physically plausible compared to existing methods.