mlsanityData to 7 Oct 2026

Unsupervised Novel View Synthesis from a Single Image

arXiv 2102.03285 ↗First posted 5 Feb 2021cs.CVcited by 8

Not on the map. Why?

Abstract (arXiv)

Novel view synthesis from a single image has recently achieved remarkable results, although the requirement of some form of 3D, pose, or multi-view supervision at training time limits the deployment in real scenarios. This work aims at relaxing these assumptions enabling training of conditional generative models for novel view synthesis in a completely unsupervised manner. We first pre-train a purely generative decoder model using a 3D-aware GAN formulation while at the same time train an encoder network to invert the mapping from latent space to images. Then, we swap encoder and decoder and train the network as a conditioned GAN with a mixture of an autoencoder-like objective and self-distillation. At test time, given a view of an object, our model first embeds the image content in a latent code and regresses its pose, then generates novel views of it by keeping the code fixed and varying the pose. We test our framework on both synthetic datasets such as ShapeNet and on unconstrained collections of natural images, where no competing methods can be trained.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 56 of its 81 references that are in the map. First posted on arXiv 5 Feb 2021.

Builds on

56 of 81 references.

References · 56

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  3. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  4. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
  5. 2013Auto-Encoding Variational Bayescited by 3,980
  6. 2014Deep Learning Face Attributes in the Wildcited by 3,679
  7. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  8. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  9. 2017Improved Training of Wasserstein GANscited by 2,223
  10. 2018Spectral Normalization for Generative Adversarial Networkscited by 1,618
  11. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,574
  12. 2019Semantic Image Synthesis with Spatially-Adaptive Normalizationcited by 1,295
38 more
  1. 2016Least Squares Generative Adversarial Networkscited by 1,000
  2. 2019Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representationscited by 864
  3. 2020GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fieldscited by 784
  4. 2019Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervisioncited by 699
  5. 2020GRAF: Generative Radiance Fields for 3D-Aware Image Synthesiscited by 688
  6. 2019Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?cited by 675
  7. 20163D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstructioncited by 587
  8. 2018DeepVoxels: Learning Persistent 3D Feature Embeddingscited by 401
  9. 2019Image2StyleGAN++: How to Edit the Embedded Images?cited by 356
  10. 2016Generative Visual Manipulation on the Natural Image Manifoldcited by 348
  11. 2020StyleSpace Analysis: Disentangled Controls for StyleGAN Image Generationcited by 331
  12. 2019DeepView: View Synthesis with Learned Gradient Descentcited by 253
  13. 2019Texture Fields: Learning Texture Representations in Function Spacecited by 210
  14. 2015DeepStereo: Learning to Predict New Views from the World's Imagerycited by 185
  15. 2019Unsupervised Learning of Probably Symmetric Deformable 3D Objects from Images in the Wildcited by 180
  16. 2016View Synthesis by Appearance Flowcited by 172
  17. 2018Escaping Plato's Cave: 3D Shape From Adversarial Renderingcited by 170
  18. 2020BlockGAN: Learning 3D Object-aware Scene Representations from Unlabelled Imagescited by 166
  19. 2016Inverting The Generator Of A Generative Adversarial Networkcited by 140
  20. 2017Learning Efficient Point Cloud Generation for Dense 3D Object Reconstructioncited by 133
  21. 2015A Large-Scale Car Dataset for Fine-Grained Categorization and Verificationcited by 129
  22. 2018Visual Object Networks: Image Generation with Disentangled 3D Representationcited by 124
  23. 2019Towards Unsupervised Learning of Generative Models for 3D Controllable Image Synthesiscited by 104
  24. 2020Generative Hierarchical Features from Synthesizing Imagescited by 83
  25. 2018Unsupervised Geometry-Aware Representation for 3D Human Pose Estimationcited by 73
  26. 2016Deep3D: Fully Automatic 2D-to-3D Video Conversion with Deep Convolutional Neural Networkscited by 72
  27. 2020Collaborative Learning for Faster StyleGAN Embeddingcited by 65
  28. 2020CONFIG: Controllable Neural Face Image Generationcited by 61
  29. 2017Transformation-Grounded Image Generation Network for Novel 3D View Synthesiscited by 59
  30. 2015Multi-view 3D Models from Single Images with a Convolutional Networkcited by 57
  31. 2018Multi-view Consistency as Supervisory Signal for Learning Shape and Pose Predictioncited by 52
  32. 2016Generative Image Modeling using Style and Structure Adversarial Networkscited by 50
  33. 2020Equivariant Neural Renderingcited by 47
  34. 2019Learning single-image 3D reconstruction by generative modelling of shape, pose and shadingcited by 43
  35. 2017Interpretable Transformations with Encoder-Decoder Networkscited by 33
  36. 2017Image2Mesh: A Learning Framework for Single Image 3D Reconstructioncited by 29
  37. 2014Learning to Generate Chairs, Tables and Cars with Convolutional Networkscited by 26
  38. 2016Weakly-supervised Disentangling with Recurrent Transformations for 3D View Synthesiscited by 24

50 most cited of 56 All 56 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 5; 2023: 1; 2024: 0; 2025: 0; 2026: 2.

8 papers.

  1. 2022Pix2NeRF: Unsupervised Conditional π-GAN for Single Image to Neural Radiance Fields Translationcited by 73
  2. 2022High-fidelity 3D GAN Inversion by Pseudo-multi-view Optimizationcited by 65
  3. 2022Shape, Pose, and Appearance from a Single Image via Bootstrapped Radiance Field Inversioncited by 30
  4. 2022LatentSwap3D: Semantic Edits on 3D Image GANscited by 5
  5. 2026FreeFix: Boosting 3D Gaussian Splatting via Fine-Tuning-Free Diffusion Modelscited by 2
  6. 2023Single-view Neural Radiance Fields with Depth Teachercited by 1
  7. 2022Multi-Frequency-Aware Patch Adversarial Learning for Neural Point Cloud Renderingcited by 0
  8. 2026One View Is Enough: In-the-Wild Monocular Pretraining for Novel View Generationcited by 0