mlsanityData to 7 Oct 2026

Pix2NeRF: Unsupervised Conditional π-GAN for Single Image to Neural Radiance Fields Translation

arXiv 2202.13162 ↗First posted 26 Feb 2022cs.CV cs.AI cs.GRcited by 73

Not on the map. Why?

Abstract (arXiv)

We propose a pipeline to generate Neural Radiance Fields~(NeRF) of an object or a scene of a specific class, conditioned on a single input image. This is a challenging task, as training NeRF requires multiple views of the same scene, coupled with corresponding poses, which are hard to obtain. Our method is based on π-GAN, a generative model for unconditional 3D-aware image synthesis, which maps random latent codes to radiance fields of a class of objects. We jointly optimize (1) the π-GAN objective to utilize its high-fidelity 3D-aware generation and (2) a carefully designed reconstruction objective. The latter includes an encoder coupled with π-GAN generator to form an auto-encoder. Unlike previous few-shot NeRF approaches, our pipeline is unsupervised, capable of being trained with independent images without 3D, multi-view, or pose supervision. Applications of our pipeline include 3d avatar generation, object-centric novel view synthesis with a single input image, and 3d-aware super-resolution, to name a few.

Key facts. Cited by 73 papers in the map (data through 7 Oct 2026). Builds on 40 of its 51 references that are in the map. First posted on arXiv 26 Feb 2022.

Builds on

40 of 51 references.

References · 40

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2014Deep Learning Face Attributes in the Wildcited by 3,679
  4. 2016Improved Techniques for Training GANscited by 2,938
  5. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  6. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,682
  7. 2017CARLA: An Open Urban Driving Simulatorcited by 2,431
  8. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,286
  9. 2020Implicit Neural Representations with Periodic Activation Functionscited by 2,214
  10. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,037
  11. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,574
  12. 2018Demystifying MMD GANscited by 1,268
28 more
  1. 2020NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collectionscited by 1,230
  2. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,204
  3. 2018Learning Implicit Fields for Generative Shape Modelingcited by 1,060
  4. 2019Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representationscited by 864
  5. 2019PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitizationcited by 818
  6. 2020GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fieldscited by 784
  7. 2021MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereocited by 735
  8. 2020Encoding in Style: a StyleGAN Encoder for Image-to-Image Translationcited by 727
  9. 2020pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesiscited by 708
  10. 2019Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervisioncited by 699
  11. 2021IBRNet: Learning Multi-View Image-Based Renderingcited by 698
  12. 2020GRAF: Generative Radiance Fields for 3D-Aware Image Synthesiscited by 688
  13. 2021StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesiscited by 477
  14. 2020MetaSDF: Meta-learning Signed Distance Functionscited by 214
  15. 2017GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibriumcited by 208
  16. 2021Stereo Radiance Fields (SRF): Learning View Synthesis for Sparse Views of Novel Scenescited by 190
  17. 2021CodeNeRF: Disentangled Neural Radiance Fields for Object Categoriescited by 183
  18. 2019Unsupervised Learning of Probably Symmetric Deformable 3D Objects from Images in the Wildcited by 180
  19. 2021NeRF--: Neural Radiance Fields Without Known Camera Parameterscited by 155
  20. 2021GNeRF: GAN-based Neural Radiance Field without Posed Cameracited by 153
  21. 2021CIPS-3D: A 3D-Aware Generator of GANs Based on Conditionally-Independent Pixel Synthesiscited by 149
  22. 2020Image GANs meet Differentiable Rendering for Inverse Graphics and Interpretable 3D Neural Renderingcited by 101
  23. 2020SDF-SRN: Learning Signed Distance 3D Object Reconstruction from Static Imagescited by 68
  24. 2015Multi-view 3D Models from Single Images with a Convolutional Networkcited by 57
  25. 2021Generative Occupancy Fields for 3D Surface-Aware Image Synthesiscited by 51
  26. 2020Equivariant Neural Renderingcited by 47
  27. 2021Learning Signal-Agnostic Manifolds of Neural Fieldscited by 45
  28. 2021Unsupervised Novel View Synthesis from a Single Imagecited by 8

Built on by

Cited by year

Citing papers per year: 2022: 20; 2023: 33; 2024: 16; 2025: 4.

73 papers.

Sort:

By area:

  1. 2023Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priorscited by 395
  2. 2023NerfDiff: Single-image View Synthesis with NeRF-guided Distillation from 3D-aware Diffusioncited by 179
  3. 2022NeRF: Neural Radiance Field in 3D Vision: A Comprehensive Review (Updated Post-Gaussian Splatting)cited by 156
  4. 2022Next3D: Generative Neural Texture Rasterization for 3D-Aware Head Avatarscited by 124
  5. 2023SINE: Semantic-driven Image-based NeRF Editing with Prior-guided Editing Fieldcited by 93
  6. 2024LN3DIFF++: Scalable Latent Neural Fields Diffusion for Speedy 3D Generationcited by 91
  7. 2022High-fidelity 3D GAN Inversion by Pseudo-multi-view Optimizationcited by 65
  8. 2022MonoNeRF: Learning a Generalizable Dynamic Radiance Field from Monocular Videoscited by 63
  9. 2023CC3D: Layout-Conditioned Generation of Compositional 3D Scenescited by 58
  10. 2022DiffDreamer: Towards Consistent Unsupervised Single-view Scene Extrapolation with Conditional Diffusion Modelscited by 51
  11. 20223D GAN Inversion with Pose Optimizationcited by 44
  12. 2023DGInStyle: Domain-Generalizable Semantic Segmentation with Image Diffusion Models and Stylized Semantic Controlcited by 44
38 more
  1. 2023ViP-NeRF: Visibility Prior for Sparse Input Neural Radiance Fieldscited by 43
  2. 20223D GAN Inversion with Facial Symmetry Priorcited by 39
  3. 2023DynPoint: Dynamic Neural Point For View Synthesiscited by 37
  4. 2024GSEdit: Efficient Text-Guided Editing of 3D Objects via Gaussian Splattingcited by 36
  5. 2023VQ3D: Learning a 3D-Aware Generative Model on ImageNetcited by 32
  6. 2022Shape, Pose, and Appearance from a Single Image via Bootstrapped Radiance Field Inversioncited by 30
  7. 2023Control3Diff: Learning Controllable 3D Diffusion Models from Single-view Imagescited by 27
  8. 2023BeyondPixels: A Comprehensive Review of the Evolution of Neural Radiance Fieldscited by 27
  9. 2023UrbanGIRAFFE: Representing Urban Scenes as Compositional Generative Neural Feature Fieldscited by 26
  10. 2023AI-Generated Content (AIGC) for Various Data Modalities: A Surveycited by 25
  11. 2025Gaussian Graph Network: Learning Efficient and Generalizable Gaussian Representations from Multi-view Imagescited by 25
  12. 2022Self-Supervised Geometry-Aware Encoder for Style-Based 3D GAN Inversioncited by 24
  13. 20233D-aware Conditional Image Synthesiscited by 24
  14. 2022NeRFInvertor: High Fidelity NeRF-GAN Inversion for Single-shot Real Image Animationcited by 23
  15. 2023NViST: In the Wild New View Synthesis from a Single Image with Transformerscited by 19
  16. 2022SPARF: Large-Scale Learning of 3D Sparse Radiance Fields from Few Input Imagescited by 17
  17. 2023Breathing New Life into 3D Assets with Generative Repaintingcited by 17
  18. 2022A Survey on Deep Generative 3D-aware Image Synthesiscited by 16
  19. 2023VIVE3D: Viewpoint-Independent Video Editing using 3D-Aware GANscited by 16
  20. 2024Pano2Room: Novel View Synthesis from a Single Indoor Panoramacited by 16
  21. 2024BeNeRF: Neural Radiance Fields from a Single Blurry Image and Event Streamcited by 15
  22. 2023HyP-NeRF: Learning Improved NeRF Priors using a HyperNetworkcited by 14
  23. 2023Point-DynRF: Point-based Dynamic Radiance Fields from a Monocular Videocited by 12
  24. 2025Neural Radiance Fields for the Real World: A Surveycited by 12
  25. 2022Generative Adversarial Networks for Image Super-Resolution: A Surveycited by 11
  26. 2023PREIM3D: 3D Consistent Precise Image Attribute Editing from a Single Imagecited by 11
  27. 2023Drone-NeRF: Efficient NeRF Based 3D Scene Reconstruction for Large-Scale Drone Surveycited by 11
  28. 2023SinMPI: Novel View Synthesis from a Single Image with Expanded Multiplane Imagescited by 11
  29. 2024GenN2N: Generative NeRF2NeRF Translationcited by 11
  30. 2023InstructPix2NeRF: Instructed 3D Portrait Editing from a Single Imagecited by 10
  31. 2024Real-time 3D-aware Portrait Editing from a Single Imagecited by 10
  32. 2023NaviNeRF: NeRF-based 3D Representation Disentanglement by Latent Semantic Navigationcited by 8
  33. 2023Multi-task Learning with 3D-Aware Regularizationcited by 8
  34. 2023WildFusion: Learning 3D-Aware Latent Diffusion Models in View Spacecited by 8
  35. 2022TT-NF: Tensor Train Neural Fieldscited by 7
  36. 2022Learning Detailed Radiance Manifolds for High-Fidelity and 3D-Consistent Portrait Synthesis from Monocular Imagecited by 7
  37. 2023Strata-NeRF : Neural Radiance Fields for Stratified Scenescited by 7
  38. 2024Text2Avatar: Text to 3D Human Avatar Generation with Codebook-Driven Body Controllable Attributecited by 6

50 most cited of 73 All 73 (JSON)