mlsanityData to 7 Oct 2026

NerfDiff: Single-image View Synthesis with NeRF-guided Distillation from 3D-aware Diffusion

arXiv 2302.10109 ↗First posted 20 Feb 2023cs.CV cs.LGcited by 179

Not on the map. Why?

Cited by 86 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Novel view synthesis from a single image requires inferring occluded regions of objects and scenes whilst simultaneously maintaining semantic and physical consistency with the input. Existing approaches condition neural radiance fields (NeRF) on local image features, projecting points to the input image plane, and aggregating 2D features to perform volume rendering. However, under severe occlusion, this projection fails to resolve uncertainty, resulting in blurry renderings that lack details. In this work, we propose NerfDiff, which addresses this issue by distilling the knowledge of a 3D-aware conditional diffusion model (CDM) into NeRF through synthesizing and refining a set of virtual views at test time. We further propose a novel NeRF-guided distillation algorithm that simultaneously generates 3D consistent virtual views from the CDM samples, and finetunes the NeRF based on the improved virtual views. Our approach significantly outperforms existing NeRF-based and geometry-free approaches on challenging datasets, including ShapeNet, ABO, and Clevr3D.

Key facts. Cited by 179 papers in the map (data through 7 Oct 2026). Builds on 57 of its 78 references that are in the map. First posted on arXiv 20 Feb 2023.

Builds on

57 of 78 references.

References · 57

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  6. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  7. 2020Denoising Diffusion Implicit Modelscited by 9,655
  8. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  9. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  10. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  11. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  12. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
38 more
  1. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
  2. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
  3. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,436
  4. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,765
  5. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,574
  6. 2022Magic3D: High-Resolution Text-to-3D Content Creationcited by 1,285
  7. 2022TensoRF: Tensorial Radiance Fieldscited by 1,275
  8. 2014Depth Map Prediction from a Single Image using a Multi-Scale Deep Networkcited by 1,264
  9. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,204
  10. 2016A Point Set Generation Network for 3D Object Reconstruction from a Single Imagecited by 972
  11. 2019Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representationscited by 864
  12. 2022Point-E: A System for Generating 3D Point Clouds from Complex Promptscited by 694
  13. 2022Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generationcited by 597
  14. 2021Zero-Shot Text-Guided Object Generation with Dream Fieldscited by 513
  15. 2021StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesiscited by 477
  16. 2017Neural 3D Mesh Renderercited by 437
  17. 2019SynSin: End-to-end View Synthesis from a Single Imagecited by 364
  18. 2021ABO: Dataset and Benchmarks for Real-World 3D Object Understandingcited by 360
  19. 2021Light Field Networks: Neural Scene Representations with Single-Evaluation Renderingcited by 330
  20. 2022Novel View Synthesis with Diffusion Modelscited by 299
  21. 20223D Neural Field Generation using Triplane Diffusioncited by 274
  22. 2020Infinite Nature: Perpetual View Generation of Natural Scenes from a Single Imagecited by 239
  23. 2021Scene Representation Transformer: Geometry-Free Novel View Synthesis Through Set-Latent Scene Representationscited by 222
  24. 2022SparseFusion: Distilling View-conditioned Diffusion for 3D Reconstructioncited by 216
  25. 2022DiffRF: Rendering-Guided 3D Radiance Field Diffusioncited by 204
  26. 2017Octree Generating Networks: Efficient Convolutional Architectures for High-resolution 3D Outputscited by 189
  27. 2020GRF: Learning a General Radiance Field for 3D Representation and Renderingcited by 183
  28. 2021CodeNeRF: Disentangled Neural Radiance Fields for Object Categoriescited by 183
  29. 2017Multi-view Supervision for Single-view Reconstruction via Differentiable Ray Consistencycited by 177
  30. 2022SinNeRF: Training Neural Radiance Fields on Complex Scenes from a Single Imagecited by 165
  31. 2022RenderDiffusion: Image Diffusion for 3D Reconstruction, Inpainting and Generationcited by 162
  32. 2022NeRDi: Single-View NeRF Synthesis with Language-Guided Diffusion as General Image Priorscited by 156
  33. 2022GAUDI: A Neural Architect for Immersive 3D Scene Generationcited by 136
  34. 2022Generalizable Patch-Based Neural Renderingcited by 121
  35. 2016Perspective Transformer Nets: Learning Single-View 3D Object Reconstruction without 3D Supervisioncited by 106
  36. 2022Vision Transformer for NeRF-Based View Synthesis from a Single Input Imagecited by 104
  37. 2021Geometry-Free View Synthesis: Transformers and no 3D Priorscited by 99
  38. 2021ShaRF: Shape-conditioned Radiance Fields from a Single Viewcited by 92

50 most cited of 57 All 57 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 62; 2024: 77; 2025: 29; 2026: 11.

179 papers.

Sort:

By area:

  1. 2023Wonder3D: Single Image to 3D using Cross-Domain Diffusioncited by 682
  2. 2023SyncDreamer: Generating Multiview-consistent Images from a Single-view Imagecited by 638
  3. 2023Directly Fine-Tuning Diffusion Models on Differentiable Rewardscited by 431
  4. 2024CAT3D: Create Anything in 3D with Multi-View Diffusion Modelscited by 425
  5. 2023Splatter Image: Ultra-Fast Single-View 3D Reconstructioncited by 359
  6. 2023ReconFusion: 3D Reconstruction with Diffusion Priorscited by 317
  7. 2024GRM: Large Gaussian Reconstruction Model for Efficient 3D Reconstruction and Generationcited by 298
  8. 2023Generative Novel View Synthesis with 3D-Aware Diffusion Modelscited by 273
  9. 2023Holodeck: Language Guided Generation of 3D Embodied AI Environmentscited by 257
  10. 2023GaussianDreamer: Fast Generation from Text to 3D Gaussians by Bridging 2D and 3D Diffusion Modelscited by 255
  11. 2023Text2Room: Extracting Textured 3D Meshes from 2D Text-to-Image Modelscited by 245
  12. 2023DMV3D: Denoising Multi-View Diffusion using 3D Large Reconstruction Modelcited by 213
38 more
  1. 20234D-fy: Text-to-4D Generation Using Hybrid Score Distillation Samplingcited by 184
  2. 2023Single-Stage Diffusion NeRF: A Unified Approach to 3D Generation and Reconstructioncited by 180
  3. 2023RichDreamer: A Generalizable Normal-Depth Diffusion Model for Detail Richness in Text-to-3Dcited by 168
  4. 2025Difix3D+: Improving 3D Reconstructions with Single-Step Diffusion Modelscited by 160
  5. 2024VD3D: Taming Large Video Diffusion Transformers for 3D Camera Controlcited by 156
  6. 2024AC3D: Analyzing and Improving 3D Camera Control in Video Diffusion Transformerscited by 151
  7. 2023State of the Art on Diffusion Models for Visual Computingcited by 126
  8. 2023Viewset Diffusion: (0-)Image-Conditioned 3D Generative Models from 2D Datacited by 113
  9. 2024Flash3D: Feed-Forward Generalisable 3D Scene Reconstruction from a Single Imagecited by 113
  10. 2023HiFA: High-fidelity Text-to-3D Generation with Advanced Diffusion Guidancecited by 109
  11. 2024RealmDreamer: Text-Driven 3D Scene Generation with Inpainting and Depth Diffusioncited by 109
  12. 2024Era3D: High-Resolution Multiview Diffusion using Efficient Row-wise Attentioncited by 99
  13. 2024Wonderland: Navigating 3D Scenes from a Single Imagecited by 98
  14. 2023BOOT: Data-free Distillation of Denoising Diffusion Models with Bootstrappingcited by 96
  15. 2023Consistent-1-to-3: Consistent Image to 3D View Synthesis via Geometry-aware Diffusion Modelscited by 79
  16. 2024NVS-Solver: Video Diffusion Model as Zero-Shot Novel View Synthesizercited by 79
  17. 2024TC4D: Trajectory-Conditioned Text-to-4D Generationcited by 76
  18. 2023EpiDiff: Enhancing Multi-View Synthesis via Localized Epipolar-Constrained Diffusioncited by 72
  19. 2023Matryoshka Diffusion Modelscited by 70
  20. 2024You See it, You Got it: Learning 3D Creation on Pose-Free Videos at Scalecited by 65
  21. 2024SPAD : Spatially Aware Multiview Diffuserscited by 62
  22. 2024World-consistent Video Diffusion with Explicit 3D Modelingcited by 62
  23. 2025Bolt3D: Generating 3D Scenes in Secondscited by 57
  24. 2024Part123: Part-aware 3D Reconstruction from a Single-view Imagecited by 54
  25. 2023RoomDreamer: Text-Driven 3D Indoor Scene Synthesis with Coherent Geometry and Texturecited by 53
  26. 2023Large-Vocabulary 3D Diffusion Model with Transformercited by 48
  27. 2024Generic 3D Diffusion Adapter Using Controlled Multi-View Editingcited by 46
  28. 2024VideoMV: Consistent Multi-View Generation Based on Large Video Generative Modelcited by 45
  29. 2023Text2Immersion: Generative Immersive Scene with 3D Gaussianscited by 43
  30. 2024PRDP: Proximal Reward Difference Prediction for Large-Scale Reward Finetuning of Diffusion Modelscited by 41
  31. 2025SPAR3D: Stable Point-Aware Reconstruction of 3D Objects from Single Imagescited by 41
  32. 2023Deceptive-NeRF/3DGS: Diffusion-Generated Pseudo-Observations for High-Quality Sparse-View Reconstructioncited by 39
  33. 2025Matrix3D: Large Photogrammetry Model All-in-Onecited by 39
  34. 2025Lyra: Generative 3D Scene Reconstruction via Video Diffusion Model Self-Distillationcited by 37
  35. 2023Tracking and Mapping in Medical Computer Vision: A Reviewcited by 36
  36. 2026Lyra 2.0: Explorable Generative 3D Worldscited by 36
  37. 2024MVD-Fusion: Single-view 3D via Depth-consistent Multi-view Generationcited by 35
  38. 2024Img2CAD: Conditioned 3D CAD Model Generation from Single Image with Structured Visual Geometrycited by 35

50 most cited of 179 All 179 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024ID-NeRF: Indirect Diffusion-guided Neural Radiance Fields for Generalizable View Synthesis
  2. 2025DT-NeRF: A Diffusion and Transformer-Based Optimization Approach for Neural Radiance Fields in 3D Reconstruction
  3. 2024DiSR-NeRF: Diffusion-Guided View-Consistent Super-Resolution NeRF
  4. 2025OccludeNeRF: Geometric-aware 3D Scene Inpainting with Collaborative Score Distillation in NeRF
  5. 2023Learning a Diffusion Prior for NeRFs
  6. 2024Taming Latent Diffusion Model for Neural Radiance Field Inpainting
  7. 2024Diffusion Priors for Dynamic View Synthesis from Monocular Videos
  8. 2024SIGNeRF: Scene Integrated Generation for Neural Radiance Fields