# Data-Driven 3D Reconstruction of Dressed Humans From Sparse Views

arXiv: https://arxiv.org/abs/2104.08013 · HTML: https://mlsanity.com/paper/2104.08013/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 74 references that are in the map.
- First posted on arXiv 16 Apr 2021.

## Builds on (most cited first, 30 of 38)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [Expressive Body Capture: 3D Hands, Face, and Body from a Single Image](https://mlsanity.com/paper/1904.05866/) (2019)
- [pixelNeRF: Neural Radiance Fields from One or Few Images](https://mlsanity.com/paper/2012.02190/) (2020)
- [Deep High-Resolution Representation Learning for Human Pose Estimation](https://mlsanity.com/paper/1902.09212/) (2019)
- [Stacked Hourglass Networks for Human Pose Estimation](https://mlsanity.com/paper/1603.06937/) (2016)
- [End-to-end Recovery of Human Shape and Pose](https://mlsanity.com/paper/1712.06584/) (2017)
- [Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representations](https://mlsanity.com/paper/1906.01618/) (2019)
- [PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization](https://mlsanity.com/paper/1905.05172/) (2019)
- [Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Image](https://mlsanity.com/paper/1607.08128/) (2016)
- [Neural Body: Implicit Neural Representations with Structured Latent Codes for Novel View Synthesis of Dynamic Humans](https://mlsanity.com/paper/2012.15838/) (2020)
- [Multi-view Convolutional Neural Networks for 3D Shape Recognition](https://mlsanity.com/paper/1505.00880/) (2015)
- [PIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitization](https://mlsanity.com/paper/2004.00452/) (2020)
- [Convolutional Mesh Regression for Single-Image Human Shape Reconstruction](https://mlsanity.com/paper/1905.03244/) (2019)
- [ARCH: Animatable Reconstruction of Clothed Humans](https://mlsanity.com/paper/2004.04572/) (2020)
- [Multi-Garment Net: Learning to Dress 3D People from Images](https://mlsanity.com/paper/1908.06903/) (2019)
- [Total Capture: A 3D Deformation Model for Tracking Faces, Hands, and Bodies](https://mlsanity.com/paper/1801.01615/) (2018)
- [Video Based Reconstruction of 3D People Models](https://mlsanity.com/paper/1803.04758/) (2018)
- [PaMIR: Parametric Model-Conditioned Implicit Representation for Image-based Human Reconstruction](https://mlsanity.com/paper/2007.03858/) (2020)
- [Learning to Estimate 3D Human Pose and Shape from a Single Color Image](https://mlsanity.com/paper/1805.04092/) (2018)
- [DeepHuman: 3D Human Reconstruction from a Single Image](https://mlsanity.com/paper/1903.06473/) (2019)
- [TailorNet: Predicting Clothing in 3D as a Function of Human Pose, Shape and Garment Style](https://mlsanity.com/paper/2003.04583/) (2020)
- [BodyNet: Volumetric Inference of 3D Human Body Shapes](https://mlsanity.com/paper/1804.04875/) (2018)
- [Combining Implicit Function Learning and Parametric Models for 3D Human Reconstruction](https://mlsanity.com/paper/2007.11432/) (2020)
- [SurfaceNet: An End-to-end 3D Neural Network for Multiview Stereopsis](https://mlsanity.com/paper/1708.01749/) (2017)
- [Geo-PIFu: Geometry and Pixel Aligned Implicit Functions for Single-view Human Reconstruction](https://mlsanity.com/paper/2006.08072/) (2020)
- [DenseRaC: Joint 3D Pose and Shape Estimation by Dense Render-and-Compare](https://mlsanity.com/paper/1910.00116/) (2019)
- [Detailed Human Shape Estimation from a Single Image by Hierarchical Mesh Deformation](https://mlsanity.com/paper/1904.10506/) (2019)

## Built on by (most cited first, 11 of 11)

- [CoNeRF: Controllable Neural Radiance Fields](https://mlsanity.com/paper/2112.01983/) (2021)
- [Geometry-Guided Progressive NeRF for Generalizable and Efficient Neural Human Rendering](https://mlsanity.com/paper/2112.04312/) (2021)
- [NSF: Neural Surface Fields for Human Modeling from Monocular Depth](https://mlsanity.com/paper/2308.14847/) (2023)
- [FLEX: Extrinsic Parameters-free Multi-view 3D Human Motion Reconstruction](https://mlsanity.com/paper/2105.01937/) (2021)
- [BodyMap: Learning Full-Body Dense Correspondence Map](https://mlsanity.com/paper/2205.09111/) (2022)
- [HDhuman: High-quality Human Novel-view Rendering from Sparse Views](https://mlsanity.com/paper/2201.08158/) (2022)
- [Super-resolution 3D Human Shape from a Single Low-Resolution Image](https://mlsanity.com/paper/2208.10738/) (2022)
- [3D Reconstruction of Interacting Multi-Person in Clothing from a Single Image](https://mlsanity.com/paper/2401.06415/) (2024)
- [COSMU: Complete 3D human shape from monocular unconstrained images](https://mlsanity.com/paper/2407.10586/) (2024)
- [NF3DM: Combining Neural Fields and Deformation Models for 3D Non-Rigid Motion Reconstruction](https://mlsanity.com/paper/2412.08511/) (2024)
- [SHARE: Single-view Human Adversarial REconstruction](https://mlsanity.com/paper/2401.00343/) (2023)

## Abstract (arXiv)

Recently, data-driven single-view reconstruction methods have shown great progress in modeling 3D dressed humans. However, such methods suffer heavily from depth ambiguities and occlusions inherent to single view inputs. In this paper, we tackle this problem by considering a small set of input views and investigate the best strategy to suitably exploit information from these views. We propose a data-driven end-to-end approach that reconstructs an implicit 3D representation of dressed humans from sparse camera views. Specifically, we introduce three key components: first a spatially consistent reconstruction that allows for arbitrary placement of the person in the input views using a perspective camera model; second an attention-based fusion layer that learns to aggregate visual information from several viewpoints; and third a mechanism that encodes local 3D patterns under the multi-view context. In the experiments, we show the proposed approach outperforms the state of the art on standard data both quantitatively and qualitatively. To demonstrate the spatially consistent reconstruction, we apply our approach to dynamic scenes. Additionally, we apply our method on real data acquired with a multi-camera platform and demonstrate our approach can obtain results comparable to multi-view stereo with dramatically less views.