mlsanityData to 14 Oct 2026

LOLNeRF: Learn from One Look

arXiv 2111.09996 ↗First posted 19 Nov 2021cs.CVcited by 84

Not on the map. Why?

Abstract (arXiv)

We present a method for learning a generative 3D model based on neural radiance fields, trained solely from data with only single views of each object. While generating realistic images is no longer a difficult task, producing the corresponding 3D structure such that they can be rendered from different views is non-trivial. We show that, unlike existing methods, one does not need multi-view data to achieve this goal. Specifically, we show that by reconstructing many images aligned to an approximate canonical pose with a single network conditioned on a shared latent space, you can learn a space of radiance fields that models shape and appearance for a class of objects. We demonstrate this by training models to reconstruct object categories using datasets that contain only one view of each subject without depth or geometry information. Our experiments show that we achieve state-of-the-art results in novel view synthesis and high-quality results for monocular depth prediction.

Key facts. Cited by 84 papers in the map (data through 14 Oct 2026). Builds on 50 of its 76 references that are in the map. First posted on arXiv 19 Nov 2021.

Builds on

50 of 76 references.

References · 50

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  3. 2014Deep Learning Face Attributes in the Wildcited by 3,693
  4. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,988
  5. 2016Improved Techniques for Training GANscited by 2,970
  6. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,689
  7. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,626
  8. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,041
  9. 2021Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fieldscited by 1,964
  10. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,584
  11. 2018Glow: Generative Flow with Invertible 1x1 Convolutionscited by 1,545
  12. 2018Demystifying MMD GANscited by 1,293
38 more
  1. 2020NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collectionscited by 1,232
  2. 2020Nerfies: Deformable Neural Radiance Fieldscited by 1,198
  3. 2019StarGAN v2: Diverse Image Synthesis for Multiple Domainscited by 1,098
  4. 2018Learning Implicit Fields for Generative Shape Modelingcited by 1,060
  5. 2019Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representationscited by 864
  6. 2020NeRF++: Analyzing and Improving Neural Radiance Fieldscited by 861
  7. 2020GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fieldscited by 785
  8. 2020pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesiscited by 708
  9. 2020GRAF: Generative Radiance Fields for 3D-Aware Image Synthesiscited by 689
  10. 2021UNISURF: Unifying Neural Implicit Surfaces and Radiance Fields for Multi-View Reconstructioncited by 646
  11. 2021BARF: Bundle-Adjusting Neural Radiance Fieldscited by 642
  12. 2021StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesiscited by 477
  13. 2020Dynamic Neural Radiance Fields for Monocular 4D Facial Avatar Reconstructioncited by 467
  14. 2021Dynamic View Synthesis from Dynamic Monocular Videocited by 420
  15. 2021Light Field Networks: Neural Scene Representations with Single-Evaluation Renderingcited by 332
  16. 20193D Morphable Face Models -- Past, Present and Futurecited by 270
  17. 2019Learning Shape Templates with Structured Implicit Functionscited by 242
  18. 2018Pros and Cons of GAN Evaluation Measurescited by 224
  19. 2020MetaSDF: Meta-learning Signed Distance Functionscited by 214
  20. 2021HyperNeRF: A Higher-Dimensional Representation for Topologically Varying Neural Radiance Fieldscited by 214
  21. 2019BSP-Net: Generating Compact Meshes via Binary Space Partitioningcited by 206
  22. 2021CodeNeRF: Disentangled Neural Radiance Fields for Object Categoriescited by 183
  23. 2019Unsupervised Learning of Probably Symmetric Deformable 3D Objects from Images in the Wildcited by 180
  24. 2019CvxNet: Learnable Convex Decompositioncited by 174
  25. 2017MoFA: Model-based Deep Convolutional Face Autoencoder for Unsupervised Monocular Reconstructioncited by 163
  26. 2017Optimizing the Latent Space of Generative Networkscited by 157
  27. 2021GNeRF: GAN-based Neural Radiance Field without Posed Cameracited by 153
  28. 2017Morphable Face Models - An Open Frameworkcited by 138
  29. 2018Discovery of Latent 3D Keypoints via End-to-end Geometric Reasoningcited by 79
  30. 2021A Shading-Guided Generative Implicit Model for Shape-Accurate 3D-Aware Image Synthesiscited by 72
  31. 2021Unsupervised Learning of 3D Object Categories from Videos in the Wildcited by 67
  32. 2021CAMPARI: Camera-Aware Decomposed Generative Neural Radiance Fieldscited by 55
  33. 2021FiG-NeRF: Figure-Ground Neural Radiance Fields for 3D Object Category Modellingcited by 47
  34. 2020Neural Volume Rendering: NeRF And Beyondcited by 44
  35. 2017Adversarial Inverse Graphics Networks: Learning 2D-to-3D Lifting and Image-to-Image Translation from Unpaired Supervisioncited by 27
  36. 2017Megapixel Size Image Creation using Generative Adversarial Networkscited by 27
  37. 2018Unsupervised Depth Estimation, 3D Face Rotation and Replacementcited by 10
  38. 2019GLOSS: Generative Latent Optimization of Sentence Representationscited by 1

Built on by

Cited by year

Citing papers per year: 2022: 25; 2023: 37; 2024: 21; 2025: 1.

84 papers.

Sort:

By area:

  1. 2022Novel View Synthesis with Diffusion Modelscited by 300
  2. 20223D Neural Field Generation using Triplane Diffusioncited by 274
  3. 2022NeuMan: Neural Human Radiance Field from a Single Videocited by 244
  4. 2023Single-Stage Diffusion NeRF: A Unified Approach to 3D Generation and Reconstructioncited by 180
  5. 2022NeRF: Neural Radiance Field in 3D Vision: A Comprehensive Review (Updated Post-Gaussian Splatting)cited by 158
  6. 2022InstantAvatar: Learning Avatars from Monocular Video in 60 Secondscited by 138
  7. 2022D2NeRF: Self-Supervised Decoupling of Dynamic and Static Objects from a Monocular Videocited by 137
  8. 2022GAUDI: A Neural Architect for Immersive 3D Scene Generationcited by 136
  9. 2022RelPose: Predicting Probabilistic Relative Rotation for Single Objects in the Wildcited by 102
  10. 2023RobustNeRF: Ignoring Distractors with Robust Lossescited by 101
  11. 2023Compositional 3D Scene Generation using Locally Conditioned Diffusioncited by 101
  12. 2024LN3DIFF++: Scalable Latent Neural Fields Diffusion for Speedy 3D Generationcited by 92
38 more
  1. 2023Bayes' Rays: Uncertainty Quantification for Neural Radiance Fieldscited by 91
  2. 2023ATT3D: Amortized Text-to-3D Object Synthesiscited by 87
  3. 2022KeypointNeRF: Generalizing Image-based Volumetric Avatars using Relative Spatial Encoding of Keypointscited by 77
  4. 2023SeaThru-NeRF: Neural Radiance Fields in Scattering Mediacited by 75
  5. 2023Real-Time Radiance Fields for Single-Image Portrait View Synthesiscited by 64
  6. 2024SpotlessSplats: Ignoring Distractors in 3D Gaussian Splattingcited by 61
  7. 20223DDesigner: Towards Photorealistic 3D Object Generation and Editing with Text-guided Diffusion Modelscited by 57
  8. 2023Learning Locally Editable Virtual Humanscited by 54
  9. 2024CAP4D: Creating Animatable 4D Portrait Avatars with Morphable Multi-View Diffusion Modelscited by 52
  10. 2023Persistent Nature: A Generative Model of Unbounded 3D Worldscited by 44
  11. 2023Alchemist: Parametric Control of Material Properties with Diffusion Modelscited by 38
  12. 2023Preface: A Data-driven Volumetric Prior for Few-shot Ultra High-resolution Face Synthesiscited by 37
  13. 2023VQ3D: Learning a 3D-Aware Generative Model on ImageNetcited by 32
  14. 2024Learning Generalizable Feature Fields for Mobile Manipulationcited by 31
  15. 2022State of the Art in Dense Monocular Non-Rigid 3D Reconstructioncited by 30
  16. 2022Shape, Pose, and Appearance from a Single Image via Bootstrapped Radiance Field Inversioncited by 30
  17. 2023HandNeRF: Neural Radiance Fields for Animatable Interacting Handscited by 27
  18. 2023Control3Diff: Learning Controllable 3D Diffusion Models from Single-view Imagescited by 27
  19. 2023BeyondPixels: A Comprehensive Review of the Evolution of Neural Radiance Fieldscited by 27
  20. 2022Self-Supervised Geometry-Aware Encoder for Style-Based 3D GAN Inversioncited by 24
  21. 2022NeRFInvertor: High Fidelity NeRF-GAN Inversion for Single-shot Real Image Animationcited by 23
  22. 2024DDN-SLAM: Real-time Dense Dynamic Neural Implicit SLAMcited by 23
  23. 2024Morphable Diffusion: 3D-Consistent Diffusion for Single-image Avatar Creationcited by 23
  24. 20223DFaceShop: Explicitly Controllable 3D-Aware Portrait Generationcited by 22
  25. 2023Robust Camera Pose Refinement for Multi-Resolution Hash Encodingcited by 21
  26. 2022Attention Beats Concatenation for Conditioning Neural Fieldscited by 20
  27. 2023FaceLit: Neural 3D Relightable Facescited by 20
  28. 2023Towards Realistic Generative 3D Face Modelscited by 20
  29. 2023Neural Image-based Avatars: Generalizable Radiance Fields for Human Avatar Modelingcited by 19
  30. 2023NViST: In the Wild New View Synthesis from a Single Image with Transformerscited by 19
  31. 2023Unsupervised Volumetric Animationcited by 18
  32. 2024SHINOBI: Shape and Illumination using Neural Object Decomposition via BRDF Optimization In-the-wildcited by 18
  33. 2022Neural Groundplans: Persistent Neural Scene Representations from a Single Imagecited by 17
  34. 2024Guess The Unseen: Dynamic 3D Scene Reconstruction from Partial 2D Glimpsescited by 17
  35. 2022A Survey on Deep Generative 3D-aware Image Synthesiscited by 16
  36. 2024BeNeRF: Neural Radiance Fields from a Single Blurry Image and Event Streamcited by 15
  37. 2024Gaussian Splatting for Efficient Satellite Image Photogrammetrycited by 15
  38. 2023HyP-NeRF: Learning Improved NeRF Priors using a HyperNetworkcited by 14

50 most cited of 84 All 84 (JSON)