ReFu: Refine and Fuse the Unobserved View for Detail-Preserving Single-Image 3D Human Reconstruction
Not on the map. Why?
Abstract (arXiv)
Single-image 3D human reconstruction aims to reconstruct the 3D textured surface of the human body given a single image. While implicit function-based methods recently achieved reasonable reconstruction performance, they still bear limitations showing degraded quality in both surface geometry and texture from an unobserved view. In response, to generate a realistic textured surface, we propose ReFu, a coarse-to-fine approach that refines the projected backside view image and fuses the refined image to predict the final human body. To suppress the diffused occupancy that causes noise in projection images and reconstructed meshes, we propose to train occupancy probability by simultaneously utilizing 2D and 3D supervisions with occupancy-based volume rendering. We also introduce a refinement architecture that generates detail-preserving backside-view images with front-to-back warping. Extensive experiments demonstrate that our method achieves state-of-the-art performance in 3D human reconstruction from a single image, showing enhanced geometry and texture quality from an unobserved view.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 48 of its 57 references that are in the map. First posted on arXiv 9 Nov 2022.
Builds on
48 of 57 references.
References · 48
- 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
- 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
- 2016Feature Pyramid Networks for Object Detectioncited by 5,021
- 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
- 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
- 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,682
- 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,037
- 2017Arbitrary Style Transfer in Real-time with Adaptive Instance Normalizationcited by 1,998
- 2016Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Networkcited by 1,898
- 2019Expressive Body Capture: 3D Hands, Face, and Body from a Single Imagecited by 1,600
- 2015Spatial Transformer Networkscited by 1,423
- 2019Semantic Image Synthesis with Spatially-Adaptive Normalizationcited by 1,295
36 more
- 2016Stacked Hourglass Networks for Human Pose Estimationcited by 913
- 2019PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitizationcited by 818
- 2018Which Training Methods for GANs do actually Converge?cited by 672
- 2018Pixel2Mesh: Generating 3D Mesh Models from Single RGB Imagescited by 649
- 2019Learning to Reconstruct 3D Human Pose and Shape via Model-fitting in the Loopcited by 625
- 2019VIBE: Video Inference for Human Body Pose and Shape Estimationcited by 615
- 2018DensePose: Dense Human Pose Estimation In The Wildcited by 573
- 2018AtlasNet: A Papier-Mach'e Approach to Learning 3D Surface Generationcited by 539
- 2020PIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitizationcited by 521
- 2019Omni-Scale Feature Learning for Person Re-Identificationcited by 292
- 2021VITON-HD: High-Resolution Virtual Try-On via Misalignment-Aware Normalizationcited by 287
- 2019Convolutional Mesh Regression for Single-Image Human Shape Reconstructioncited by 270
- 2018Video Based Reconstruction of 3D People Modelscited by 237
- 2020PaMIR: Parametric Model-Conditioned Implicit Representation for Image-based Human Reconstructioncited by 231
- 2019DeepHuman: 3D Human Reconstruction from a Single Imagecited by 182
- 2019Tex2Shape: Detailed Full Human Body Geometry From a Single Imagecited by 166
- 2018BodyNet: Volumetric Inference of 3D Human Body Shapescited by 141
- 2019Progressive Pose Attention Transfer for Person Image Generationcited by 138
- 2017Learning Efficient Point Cloud Generation for Dense 3D Object Reconstructioncited by 133
- 2020Geo-PIFu: Geometry and Pixel Aligned Implicit Functions for Single-view Human Reconstructioncited by 112
- 2018Detailed Human Avatars from Monocular Videocited by 109
- 2020Deep Image Spatial Transformation for Person Image Generationcited by 100
- 2018DoubleFusion: Real-time Capture of Human Performances with Inner Body Shapes from a Single Depth Sensorcited by 95
- 2017Towards Accurate Markerless Human Shape and Pose Estimation over Timecited by 92
- 2018DeepWrinkles: Accurate and Realistic Clothing Modelingcited by 90
- 2019360-Degree Textures of People in Clothing from a Single Imagecited by 75
- 2019Detailed Human Shape Estimation from a Single Image by Hierarchical Mesh Deformationcited by 71
- 2018SiCloPe: Silhouette-Based Clothed Peoplecited by 68
- 2019Moulding Humans: Non-parametric 3D Human Shape Estimation from Single Imagescited by 67
- 2021DeepMultiCap: Performance Capture of Multiple Characters Using Sparse Multiview Camerascited by 67
- 2020Pose-Guided Human Animation from a Single Image in the Wildcited by 53
- 2021Style and Pose Control for Image Synthesis of Humans from a Single Monocular Viewcited by 53
- 2021Generative Occupancy Fields for 3D Surface-Aware Image Synthesiscited by 51
- 2021Pose with Style: Detail-Preserving Pose-Guided Image Synthesis with Conditional StyleGANcited by 45
- 20213D Human Texture Estimation from a Single Image with Transformerscited by 35
- 20183D Human Body Reconstruction from a Single Image via Volumetric Regressioncited by 29
Built on by
Citing papers per year: 2023: 1; 2024: 0; 2025: 1.
2 papers.