mlsanityData to 14 Oct 2026

2D GANs Meet Unsupervised Single-view 3D Reconstruction

arXiv 2207.10183 ↗First posted 20 Jul 2022cs.CVcited by 11

Not on the map. Why?

Abstract (arXiv)

Recent research has shown that controllable image generation based on pre-trained GANs can benefit a wide range of computer vision tasks. However, less attention has been devoted to 3D vision tasks. In light of this, we propose a novel image-conditioned neural implicit field, which can leverage 2D supervisions from GAN-generated multi-view images and perform the single-view reconstruction of generic objects. Firstly, a novel offline StyleGAN-based generator is presented to generate plausible pseudo images with full control over the viewpoint. Then, we propose to utilize a neural implicit function, along with a differentiable renderer to learn 3D geometry from pseudo images with object masks and rough pose initializations. To further detect the unreliable supervisions, we introduce a novel uncertainty module to predict uncertainty maps, which remedy the negative effect of uncertain regions in pseudo images, leading to a better reconstruction performance. The effectiveness of our approach is demonstrated through superior single-view 3D reconstruction results of generic objects.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Builds on 61 of its 76 references that are in the map. First posted on arXiv 20 Jul 2022.

Builds on

61 of 76 references.

References · 61

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  4. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,588
  5. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,988
  6. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,626
  7. 2017What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?cited by 2,352
  8. 2015LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loopcited by 1,355
  9. 2020Training Generative Adversarial Networks with Limited Datacited by 1,186
  10. 2021Alias-Free Generative Adversarial Networkscited by 1,184
  11. 2019FaceForensics++: Learning to Detect Manipulated Facial Imagescited by 1,101
  12. 2016HyperNetworkscited by 1,027
38 more
  1. 2019Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representationscited by 864
  2. 2020GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fieldscited by 785
  3. 2020Implicit Geometric Regularization for Learning Shapescited by 710
  4. 2020pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesiscited by 708
  5. 2019Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervisioncited by 700
  6. 2020GRAF: Generative Radiance Fields for 3D-Aware Image Synthesiscited by 689
  7. 2020GANSpace: Discovering Interpretable GAN Controlscited by 614
  8. 2016Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modelingcited by 558
  9. 2017MoCoGAN: Decomposing Motion and Content for Video Generationcited by 524
  10. 2021StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesiscited by 477
  11. 2020U2-Net: Going Deeper with Nested U-Structure for Salient Object Detectioncited by 412
  12. 2020Closed-Form Factorization of Latent Semantics in GANscited by 408
  13. 2020InterFaceGAN: Interpreting the Disentangled Face Representation Learned by GANscited by 394
  14. 2020StyleFlow: Attribute-conditioned Exploration of StyleGAN-Generated Images using Conditional Continuous Normalizing Flowscited by 349
  15. 2021Towards Real-World Blind Face Restoration with Generative Facial Priorcited by 349
  16. 2018Learning Category-Specific Mesh Reconstruction from Image Collectionscited by 317
  17. 2021DatasetGAN: Efficient Labeled Data Factory with Minimal Human Effortcited by 264
  18. 2019On the Detection of Digital Face Manipulationcited by 260
  19. 2020Exploiting Deep Generative Prior for Versatile Image Restoration and Manipulationcited by 225
  20. 2021GAN Prior Embedded Network for Blind Face Restoration in the Wildcited by 197
  21. 2017Learning a Multi-View Stereo Machinecited by 194
  22. 2019Image Processing Using Multi-Code GAN Priorcited by 185
  23. 2019Unsupervised Learning of Probably Symmetric Deformable 3D Objects from Images in the Wildcited by 180
  24. 2019DIST: Rendering Deep Implicit Signed Distance Function with Differentiable Sphere Tracingcited by 180
  25. 2017Multi-view Supervision for Single-view Reconstruction via Differentiable Ray Consistencycited by 177
  26. 2019SDFDiff: Differentiable Rendering of Signed Distance Fields for 3D Shape Optimizationcited by 159
  27. 2020Self-supervised Single-view 3D Reconstruction via Semantic Consistencycited by 127
  28. 2018Visual Object Networks: Image Generation with Disentangled 3D Representationcited by 124
  29. 2020Image GANs meet Differentiable Rendering for Inverse Graphics and Interpretable 3D Neural Renderingcited by 101
  30. 2020Shape and Viewpoint without Keypointscited by 92
  31. 2020Articulation-aware Canonical Surface Mappingcited by 81
  32. 2020SDF-SRN: Learning Signed Distance 3D Object Reconstruction from Static Imagescited by 68
  33. 2021Shelf-Supervised Mesh Prediction in the Wildcited by 65
  34. 2020Lifting 2D StyleGAN for 3D-Aware Face Generationcited by 64
  35. 2019Canonical Surface Mapping via Geometric Cycle Consistencycited by 63
  36. 2020Do 2D GANs Know 3D Shape? Unsupervised 3D shape reconstruction from 2D Image GANscited by 63
  37. 2021Reverse Engineering of Generative Models: Inferring Model Hyperparameters from Generated Imagescited by 58
  38. 2019Unsupervised Generative 3D Shape Learning from Natural Imagescited by 57

50 most cited of 61 All 61 (JSON)

Built on by

Cited by year

Citing papers per year: 2022: 2; 2023: 8; 2024: 1.

11 papers.

  1. 2022SceneRF: Self-Supervised Monocular 3D Scene Reconstruction with Radiance Fieldscited by 49
  2. 2023Tame a Wild Camera: In-the-Wild Monocular Camera Calibrationcited by 34
  3. 2023Learning Clothing and Pose Invariant 3D Shape Representation for Long-Term Person Re-Identificationcited by 34
  4. 2023Lift3D: Synthesize 3D Training Data by Lifting 2D GAN to 3D Generative Radiance Fieldcited by 15
  5. 2023NoisyTwins: Class-Consistent and Diverse Image Generation through StyleGANscited by 12
  6. 2023Progressive Learning of 3D Reconstruction Network from 2D GAN Datacited by 10
  7. 2023A Review of Deep Learning-Powered Mesh Reconstruction Methodscited by 3
  8. 2024GenUDC: High Quality 3D Mesh Generation with Unsigned Dual Contouring Representationcited by 3
  9. 2022Learning Implicit Functions for Dense 3D Shape Correspondence of Generic Objectscited by 2
  10. 20233D Semantic Subspace Traverser: Empowering 3D Generative Model with Shape Editing Capabilitycited by 1
  11. 2023Self-supervised Learning for Enhancing Geometrical Modeling in 3D-Aware Generative Adversarial Networkcited by 1