mlsanityData to 7 Oct 2026

Learning a Diffusion Prior for NeRFs

arXiv 2304.14473 ↗First posted 27 Apr 2023cs.CV cs.AI cs.LGcited by 9

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

Neural Radiance Fields (NeRFs) have emerged as a powerful neural 3D representation for objects and scenes derived from 2D data. Generating NeRFs, however, remains difficult in many scenarios. For instance, training a NeRF with only a small number of views as supervision remains challenging since it is an under-constrained problem. In such settings, it calls for some inductive prior to filter out bad local minima. One way to introduce such inductive priors is to learn a generative model for NeRFs modeling a certain class of scenes. In this paper, we propose to use a diffusion model to generate NeRFs encoded on a regularized grid. We show that our model can sample realistic NeRFs, while at the same time allowing conditional generations, given a certain observation as guidance.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 35 of its 38 references that are in the map. First posted on arXiv 27 Apr 2023.

Builds on

35 of 38 references.

Closest ancestors · 3

Same direction, newest first

  1. 20233DShape2VecSet: A 3D Shape Representation for Neural Fields and Generative Diffusion Models26 Jan
  2. 2022DiffRF: Rendering-Guided 3D Radiance Field Diffusion2 Dec
  3. 2022DreamFusion: Text-to-3D using 2D Diffusion29 Sep

Other references · 32

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  4. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  5. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  6. 2022Generative Adversarial Networkscited by 5,603
  7. 2013Auto-Encoding Variational Bayescited by 3,980
  8. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
  9. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,698
  10. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  11. 2022Video Diffusion Modelscited by 2,280
  12. 2021Plenoxels: Radiance Fields without Neural Networkscited by 1,607
20 more
  1. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,574
  2. 2021Image Super-Resolution via Iterative Refinementcited by 1,506
  3. 2021Palette: Image-to-Image Diffusion Modelscited by 1,296
  4. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,204
  5. 2021Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstructioncited by 961
  6. 2019Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representationscited by 864
  7. 2020GRAF: Generative Radiance Fields for 3D-Aware Image Synthesiscited by 688
  8. 2021RegNeRF: Regularizing Neural Radiance Fields for View Synthesis from Sparse Inputscited by 614
  9. 20213D Shape Generation and Completion through Point-Voxel Diffusioncited by 516
  10. 2019PointFlow: 3D Point Cloud Generation with Continuous Normalizing Flowscited by 487
  11. 2020Learning Gradient Fields for Shape Generationcited by 245
  12. 2022Panoptic Neural Fields: A Semantic Object-Aware Neural Scene Representationcited by 243
  13. 2020Learned Initializations for Optimizing Coordinate-Based Neural Representationscited by 240
  14. 2022Diffusion models as plug-and-play priorscited by 230
  15. 2021CodeNeRF: Disentangled Neural Radiance Fields for Object Categoriescited by 183
  16. 2022GAUDI: A Neural Architect for Immersive 3D Scene Generationcited by 136
  17. 2021NeRF-VAE: A Geometry Aware 3D Scene Generative Modelcited by 111
  18. 2022EpiGRAF: Rethinking training of 3D GANscited by 108
  19. 2022ReLU Fields: The Little Non-linearity That Couldcited by 87
  20. 2022ProbNeRF: Uncertainty-Aware Inference of 3D Shapes from 2D Imagescited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 3; 2025: 1; 2026: 2.

9 papers.

  1. 2024AGG: Amortized Generative 3D Gaussians for Single Image to 3Dcited by 51
  2. 2023Ghost on the Shell: An Expressive Representation of General 3D Shapescited by 24
  3. 2023HumanLiff: Layer-wise 3D Human Generation with Diffusion Modelcited by 23
  4. 2024NeRF-US: Removing Ultrasound Imaging Artifacts from Neural Radiance Fields in the Wildcited by 7
  5. 2023RecFusion: A Binomial Diffusion Process for 1D Data for Recommendationcited by 6
  6. 2024Toward General Object-level Mapping from Sparse Views with 3D Diffusion Priorscited by 5
  7. 2025Spatial-Spectral Diffusion Contrastive Representation Network for Hyperspectral Image Classificationcited by 1
  8. 2026Predicting 3D structure by latent posterior samplingcited by 1
  9. 2026Neural Tree Reconstruction for the Open Forest Observatorycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024ID-NeRF: Indirect Diffusion-guided Neural Radiance Fields for Generalizable View Synthesis
  2. 2024Taming Latent Diffusion Model for Neural Radiance Field Inpainting
  3. 2023DiffusioNeRF: Regularizing Neural Radiance Fields with Denoising Diffusion Models
  4. 2023Single-Stage Diffusion NeRF: A Unified Approach to 3D Generation and Reconstruction
  5. 2023RePaint-NeRF: NeRF Editting via Semantic Masks and Diffusion Models
  6. 2024Generative Lifting of Multiview to 3D from Unknown Pose: Wrapping NeRF inside Diffusion
  7. 2025DT-NeRF: A Diffusion and Transformer-Based Optimization Approach for Neural Radiance Fields in 3D Reconstruction
  8. 2023NerfDiff: Single-image View Synthesis with NeRF-guided Distillation from 3D-aware Diffusion