mlsanityData to 7 Oct 2026

DiffRF: Rendering-Guided 3D Radiance Field Diffusion

arXiv 2212.01206 ↗First posted 2 Dec 2022cs.CVcited by 204

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

We introduce DiffRF, a novel approach for 3D radiance field synthesis based on denoising diffusion probabilistic models. While existing diffusion-based methods operate on images, latent codes, or point cloud data, we are the first to directly generate volumetric radiance fields. To this end, we propose a 3D denoising model which directly operates on an explicit voxel grid representation. However, as radiance fields generated from a set of posed images can be ambiguous and contain artifacts, obtaining ground truth radiance field samples is non-trivial. We address this challenge by pairing the denoising formulation with a rendering loss, enabling our model to learn a deviated prior that favours good image quality instead of trying to replicate fitting errors like floating artifacts. In contrast to 2D-diffusion models, our model learns multi-view consistent priors, enabling free-view synthesis and accurate shape generation. Compared to 3D GANs, our diffusion-based approach naturally enables conditional generation such as masked completion or single-view 3D synthesis at inference time.

Key facts. Cited by 204 papers in the map (data through 7 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 68 of its 83 references that are in the map. First posted on arXiv 2 Dec 2022.

Builds on

68 of 83 references.

Closest ancestors · 6

Same direction, newest first

  1. 20223DDesigner: Towards Photorealistic 3D Object Generation and Editing with Text-guided Diffusion Models25 Nov
  2. 2022RenderDiffusion: Image Diffusion for 3D Reconstruction, Inpainting and Generation17 Nov
  3. 2022LION: Latent Point Diffusion Models for 3D Shape Generation12 Oct
  4. 2022DreamFusion: Text-to-3D using 2D Diffusion29 Sep
  5. 2022RePaint: Inpainting using Denoising Diffusion Probabilistic Models24 Jan
  6. 2021Diffusion Probabilistic Models for 3D Point Cloud Generation2 Mar

Other references · 62

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2020Denoising Diffusion Implicit Modelscited by 9,655
  4. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  5. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  6. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  7. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  8. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  9. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
  10. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  11. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
  12. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 2,994
38 more
  1. 2016Improved Techniques for Training GANscited by 2,938
  2. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  3. 2022Video Diffusion Modelscited by 2,280
  4. 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,903
  5. 2018Spectral Normalization for Generative Adversarial Networkscited by 1,618
  6. 2021Plenoxels: Radiance Fields without Neural Networkscited by 1,607
  7. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,574
  8. 2022TensoRF: Tensorial Radiance Fieldscited by 1,275
  9. 2018Demystifying MMD GANscited by 1,268
  10. 2020NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collectionscited by 1,230
  11. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,204
  12. 2020Improved Techniques for Training Score-Based Generative Modelscited by 1,036
  13. 2021Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstructioncited by 961
  14. 2020GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fieldscited by 784
  15. 2021Resolution-robust Large Mask Inpainting with Fourier Convolutionscited by 730
  16. 2022Block-NeRF: Scalable Large Scene Neural View Synthesiscited by 720
  17. 2020pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesiscited by 708
  18. 2021IBRNet: Learning Multi-View Image-Based Renderingcited by 698
  19. 2020GRAF: Generative Radiance Fields for 3D-Aware Image Synthesiscited by 688
  20. 2021Maximum Likelihood Training of Score-Based Diffusion Modelscited by 670
  21. 2018Generative Image Inpainting with Contextual Attentioncited by 578
  22. 2016Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modelingcited by 558
  23. 20213D Shape Generation and Completion through Point-Voxel Diffusioncited by 516
  24. 2022GeoDiff: a Geometric Diffusion Model for Molecular Conformation Generationcited by 511
  25. 2019PointFlow: 3D Point Cloud Generation with Continuous Normalizing Flowscited by 487
  26. 2021StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesiscited by 477
  27. 2022GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Imagescited by 446
  28. 2021Neural Geometric Level of Detail: Real-time Rendering with Implicit 3D Shapescited by 426
  29. 2021Mega-NeRF: Scalable Construction of Large-Scale NeRFs for Virtual Fly-Throughscited by 396
  30. 2021ABO: Dataset and Benchmarks for Real-World 3D Object Understandingcited by 360
  31. 2020Learning Gradient Fields for Shape Generationcited by 245
  32. 2022Panoptic Neural Fields: A Semantic Object-Aware Neural Scene Representationcited by 243
  33. 2022Diffusion Probabilistic Modeling for Video Generationcited by 237
  34. 2022Diffusion probabilistic modeling of protein backbones in 3D for the motif-scaffolding problemcited by 229
  35. 2018Pix3D: Dataset and Methods for Single-Image 3D Shape Modelingcited by 218
  36. 2021A Variational Perspective on Diffusion-Based Generative Models and Score Matchingcited by 182
  37. 2018Escaping Plato's Cave: 3D Shape From Adversarial Renderingcited by 170
  38. 2021On Fast Sampling of Diffusion Probabilistic Modelscited by 170

50 most cited of 62 All 62 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 64; 2024: 80; 2025: 40; 2026: 20.

204 papers.

Sort:

By area:

  1. 2023ScanNet++: A High-Fidelity Dataset of 3D Indoor Scenescited by 747
  2. 2024Structured 3D Latents for Scalable and Versatile 3D Generationcited by 742
  3. 2024LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creationcited by 730
  4. 2023Wonder3D: Single Image to 3D using Cross-Domain Diffusioncited by 682
  5. 2023SyncDreamer: Generating Multiview-consistent Images from a Single-view Imagecited by 638
  6. 2023MultiDiffusion: Fusing Diffusion Paths for Controlled Image Generationcited by 529
  7. 2024InstantMesh: Efficient 3D Mesh Generation from a Single Image with Sparse-view Large Reconstruction Modelscited by 421
  8. 2023Splatter Image: Ultra-Fast Single-View 3D Reconstructioncited by 359
  9. 2023GaussianDreamer: Fast Generation from Text to 3D Gaussians by Bridging 2D and 3D Diffusion Modelscited by 255
  10. 2023MeshGPT: Generating Triangle Meshes with Decoder-Only Transformerscited by 254
  11. 2023Text2Tex: Text-driven Texture Synthesis via Diffusion Modelscited by 234
  12. 2023Single-Stage Diffusion NeRF: A Unified Approach to 3D Generation and Reconstructioncited by 180
38 more
  1. 2023NerfDiff: Single-image View Synthesis with NeRF-guided Distillation from 3D-aware Diffusioncited by 179
  2. 2023RichDreamer: A Generalizable Normal-Depth Diffusion Model for Detail Richness in Text-to-3Dcited by 168
  3. 2025Native and Compact Structured Latents for 3D Generationcited by 157
  4. 2023HyperDiffusion: Generating Implicit Neural Fields with Weight-Space Diffusioncited by 152
  5. 2023State of the Art on Diffusion Models for Visual Computingcited by 126
  6. 2024latentSplat: Autoencoding Variational Gaussians for Fast Generalizable 3D Reconstructioncited by 114
  7. 2023Viewset Diffusion: (0-)Image-Conditioned 3D Generative Models from 2D Datacited by 113
  8. 2023Diffusion with Forward Models: Solving Stochastic Inverse Problems Without Direct Supervisioncited by 112
  9. 20244Diffusion: Multi-view Video Diffusion Model for 4D Generationcited by 104
  10. 20243DTopia-XL: Scaling High-quality 3D Asset Generation via Primitive Diffusioncited by 100
  11. 2023Global Structure-Aware Diffusion Process for Low-Light Image Enhancementcited by 92
  12. 2024LN3DIFF++: Scalable Latent Neural Fields Diffusion for Speedy 3D Generationcited by 91
  13. 2025MEt3R: Measuring Multi-View Consistency in Generated Imagescited by 91
  14. 2023EMDM: Efficient Motion Diffusion Model for Fast and High-Quality Motion Generationcited by 88
  15. 2024EdgeRunner: Auto-regressive Auto-encoder for Artistic Mesh Generationcited by 84
  16. 2024BlockFusion: Expandable 3D Scene Generation using Latent Tri-plane Extrapolationcited by 81
  17. 2023Nerfbusters: Removing Ghostly Artifacts from Casually Captured NeRFscited by 76
  18. 2023EpiDiff: Enhancing Multi-View Synthesis via Localized Epipolar-Constrained Diffusioncited by 72
  19. 20243DTopia: Large Text-to-3D Generation Model with Hybrid Diffusion Priorscited by 72
  20. 2024Scaling Mesh Generation via Compressive Tokenizationcited by 71
  21. 20233D-aware Image Generation using 2D Diffusion Modelscited by 69
  22. 2024MultiDiff: Consistent Novel View Synthesis from a Single Imagecited by 67
  23. 2024Meta 3D AssetGen: Text-to-Mesh Generation with High-Quality Geometry, Texture, and PBR Materialscited by 66
  24. 2024Advances in 3D Generation: A Surveycited by 62
  25. 2024ViewDiff: 3D-Consistent Image Generation with Text-to-Image Modelscited by 57
  26. 2025Bolt3D: Generating 3D Scenes in Secondscited by 57
  27. 2023ControlRoom3D: Room Generation using Semantic Proxy Roomscited by 54
  28. 2023Improving 3D Imaging with Pre-Trained Perpendicular 2D Diffusion Modelscited by 53
  29. 2023RoomDreamer: Text-Driven 3D Indoor Scene Synthesis with Coherent Geometry and Texturecited by 53
  30. 2025Amodal3R: Amodal 3D Reconstruction from Occluded 2D Imagescited by 52
  31. 2023AutoDecoding Latent 3D Diffusion Modelscited by 50
  32. 2024DiffGS: Functional Gaussian Splatting Diffusioncited by 50
  33. 2025SceneGen: Single-Image 3D Scene Generation in One Feedforward Passcited by 49
  34. 2023Large-Vocabulary 3D Diffusion Model with Transformercited by 48
  35. 2023Mosaic-SDF for 3D Generative Modelscited by 48
  36. 2024PivotMesh: Generic 3D Mesh Generation via Pivot Vertices Guidancecited by 47
  37. 2025Efficient Part-level 3D Object Generation via Dual Volume Packingcited by 47
  38. 2024Generic 3D Diffusion Adapter Using Controlled Multi-View Editingcited by 46

50 most cited of 204 All 204 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023DiffusioNeRF: Regularizing Neural Radiance Fields with Denoising Diffusion Models
  2. 2024Diffusion-based Light Field Synthesis
  3. 2024Taming Latent Diffusion Model for Neural Radiance Field Inpainting
  4. 2025Difix3D+: Improving 3D Reconstructions with Single-Step Diffusion Models
  5. 2024Drantal-NeRF: Diffusion-Based Restoration for Anti-aliasing Neural Radiance Field
  6. 2023Optimized View and Geometry Distillation from Multi-view Diffuser
  7. 2023Deceptive-NeRF/3DGS: Diffusion-Generated Pseudo-Observations for High-Quality Sparse-View Reconstruction
  8. 2023HiFA: High-fidelity Text-to-3D Generation with Advanced Diffusion Guidance