mlsanityData to 7 Oct 2026

NeRFiller: Completing Scenes via Generative 3D Inpainting

arXiv 2312.04560 ↗First posted 7 Dec 2023cs.CV cs.AI cs.GRcited by 62

Not on the map. Why?

Abstract (arXiv)

We propose NeRFiller, an approach that completes missing portions of a 3D capture via generative 3D inpainting using off-the-shelf 2D visual generative models. Often parts of a captured 3D scene or object are missing due to mesh reconstruction failures or a lack of observations (e.g., contact regions, such as the bottom of objects, or hard-to-reach areas). We approach this challenging 3D inpainting problem by leveraging a 2D inpainting diffusion model. We identify a surprising behavior of these models, where they generate more 3D consistent inpaints when images form a 2×2 grid, and show how to generalize this behavior to more than four images. We then present an iterative framework to distill these inpainted regions into a single consistent 3D scene. In contrast to related works, we focus on completing scenes rather than deleting foreground objects, and our approach does not require tight 2D object masks or text. We compare our approach to relevant baselines adapted to our setting on a variety of scenes, where NeRFiller creates the most 3D consistent and plausible scene completions. Our project page is at https://ethanweber.me/nerfiller.

Key facts. Cited by 62 papers in the map (data through 7 Oct 2026). Builds on 54 of its 64 references that are in the map. First posted on arXiv 7 Dec 2023.

Builds on

54 of 64 references.

References · 54

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  4. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
  5. 2022Classifier-Free Diffusion Guidancecited by 5,717
  6. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,286
  7. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
  8. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,591
  9. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,231
  10. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,522
  11. 2022RePaint: Inpainting using Denoising Diffusion Probabilistic Modelscited by 1,419
  12. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillationcited by 1,207
38 more
  1. 2021MUSIQ: Multi-scale Image Quality Transformercited by 1,058
  2. 2021LoFTR: Detector-Free Local Feature Matching with Transformerscited by 974
  3. 2022Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generationcited by 958
  4. 2023Objaverse-XL: A Universe of 10M+ 3D Objectscited by 767
  5. 2021Resolution-robust Large Mask Inpainting with Fourier Convolutionscited by 730
  6. 2019Local Light Field Fusion: Practical View Synthesis with Prescriptive Sampling Guidelinescited by 672
  7. 2023ZoeDepth: Zero-shot Transfer by Combining Relative and Metric Depthcited by 646
  8. 2022Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generationcited by 597
  9. 2023Nerfstudio: A Modular Framework for Neural Radiance Field Developmentcited by 596
  10. 2023MultiDiffusion: Fusing Diffusion Paths for Controlled Image Generationcited by 529
  11. 2023Instruct-NeRF2NeRF: Editing 3D Scenes with Instructionscited by 482
  12. 2023TokenFlow: Consistent Diffusion Features for Consistent Video Editingcited by 440
  13. 2019SynSin: End-to-end View Synthesis from a Single Imagecited by 364
  14. 2023RealFusion: 360deg Reconstruction of Any Object from a Single Imagecited by 305
  15. 2022Novel View Synthesis with Diffusion Modelscited by 299
  16. 2023Generative Novel View Synthesis with 3D-Aware Diffusion Modelscited by 273
  17. 2020Single-View View Synthesis with Multiplane Imagescited by 263
  18. 2022Visual Prompting via Image Inpaintingcited by 248
  19. 2023SparseNeRF: Distilling Depth Ranking for Few-shot Novel View Synthesiscited by 247
  20. 2023Text2Room: Extracting Textured 3D Meshes from 2D Text-to-Image Modelscited by 245
  21. 2020Infinite Nature: Perpetual View Generation of Natural Scenes from a Single Imagecited by 239
  22. 2023DreamBooth3D: Subject-Driven Text-to-3D Generationcited by 235
  23. 2022SparseFusion: Distilling View-conditioned Diffusion for 3D Reconstructioncited by 216
  24. 2023MVDiffusion: Enabling Holistic Multi-view Image Generation with Correspondence-Aware Diffusioncited by 210
  25. 2023SceneScape: Text-Driven Consistent Scene Generationcited by 160
  26. 2023DreamCraft3D: Hierarchical 3D Generation with Bootstrapped Diffusion Priorcited by 157
  27. 2022SPIn-NeRF: Multiview Segmentation and Perceptual Inpainting with Neural Radiance Fieldscited by 144
  28. 2021Unconstrained Scene Generation with Locally Conditioned Radiance Fieldscited by 130
  29. 2023State of the Art on Diffusion Models for Visual Computingcited by 126
  30. 2021Pathdreamer: A World Model for Indoor Navigationcited by 117
  31. 2023Bayes' Rays: Uncertainty Quantification for Neural Radiance Fieldscited by 90
  32. 2021PixelSynth: Generating a 3D-Consistent Experience from a Single Imagecited by 82
  33. 2022InfiniteNature-Zero: Learning Perpetual View Generation of Natural Scenes from Single Imagescited by 82
  34. 2023PanoGen: Text-Conditioned Panoramic Environment Generation for Vision-and-Language Navigationcited by 77
  35. 2023Nerfbusters: Removing Ghostly Artifacts from Casually Captured NeRFscited by 76
  36. 2023SceneDreamer: Unbounded 3D Scene Generation from 2D Image Collectionscited by 72
  37. 2022Removing Objects From Neural Radiance Fieldscited by 66
  38. 2023RealFill: Reference-Driven Generation for Authentic Image Completioncited by 58

50 most cited of 54 All 54 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 18; 2025: 36; 2026: 8.

62 papers.

Sort:

By area:

  1. 2024CAT3D: Create Anything in 3D with Multi-View Diffusion Modelscited by 425
  2. 2024Stereo4D: Learning How Things Move in 3D from Internet Stereo Videoscited by 89
  3. 2024FlashTex: Fast Relightable Mesh Texturing with LightControlNetcited by 56
  4. 2024InFusion: Inpainting 3D Gaussians via Learning Depth Completion from Diffusion Priorcited by 55
  5. 2024Instant3dit: Multiview Inpainting for Fast Editing of 3D Objectscited by 43
  6. 2024MVInpainter: Learning Multi-View Consistent Inpainting to Bridge 2D and 3D Editingcited by 30
  7. 2025Decompositional Neural Scene Reconstruction with Generative Diffusion Priorcited by 30
  8. 2024Gaussian Splatting on the Move: Blur and Rolling Shutter Compensation for Natural Camera Motioncited by 29
  9. 20253D Gaussian Inpainting with Depth-Guided Cross-View Consistencycited by 23
  10. 2024RefFusion: Reference Adapted Diffusion Models for 3D Scene Inpaintingcited by 21
  11. 2025AuraFusion360: Augmented Unseen Region Alignment for Reference-based 360deg Unbounded Scene Inpaintingcited by 21
  12. 2024Consolidating Attention Features for Multi-view Image Editingcited by 19
38 more
  1. 2025EditP23: 3D Editing via Propagation of Image Prompts to Multi-Viewcited by 19
  2. 2025Viser: Imperative, Web-based 3D Visualization in Pythoncited by 19
  3. 20243D Mesh Editing using Masked LRMscited by 17
  4. 2025Orchid: Image Latent Diffusion for Joint Appearance and Geometry Generationcited by 17
  5. 2025Skyfall-GS: Synthesizing Immersive 3D Urban Scenes from Satellite Imagerycited by 16
  6. 2025Reconstruct, Inpaint, Test-Time Finetune: Dynamic Novel-view Synthesis from Monocular Videoscited by 13
  7. 2025InstaScene: Towards Complete 3D Instance Decomposition and Reconstruction from Cluttered Scenescited by 11
  8. 2025IMFine: 3D Inpainting via Geometry-guided Multi-view Refinementcited by 10
  9. 2025G4Splat: Geometry-Guided Gaussian Splatting with Generative Priorcited by 10
  10. 2025InstaInpaint: Instant 3D-Scene Inpainting with Masked Large Reconstruction Modelcited by 9
  11. 2025Inpaint360GS: Efficient Object-Aware 3D Inpainting via Gaussian Splatting for 360deg Scenescited by 8
  12. 2024View-consistent Object Removal in Radiance Fieldscited by 7
  13. 2025HairCUP: Hair Compositional Universal Prior for 3D Gaussian Avatarscited by 7
  14. 2025Geometry-Aware Diffusion Models for Multiview Scene Inpaintingcited by 6
  15. 2024SimVS: Simulating World Inconsistencies for Robust View Synthesiscited by 5
  16. 2026OpenVoxel: Training-Free Grouping and Captioning Voxels for Open-Vocabulary 3D Scene Understandingcited by 5
  17. 20243D StreetUnveiler with Semantic-aware 2DGS -- a simple baselinecited by 4
  18. 2025Coupled Diffusion Sampling for Training-Free Multi-View Image Editingcited by 4
  19. 2025GENA3D: Generative Amodal 3D Modeling by Bridging 2D Priors and 3D Coherencecited by 4
  20. 2025GaMO: Geometry-aware Multi-view Diffusion Outpainting for Sparse-View 3D Reconstructioncited by 4
  21. 2024Towards a Science Exocortexcited by 3
  22. 2025DiGA3D: Coarse-to-Fine Diffusional Propagation of Geometry and Appearance for Versatile 3D Inpaintingcited by 3
  23. 2025Affine-Equivariant Kernel Space Encoding for NeRF Editingcited by 3
  24. 2025FixingGS: Enhancing 3D Gaussian Splatting via Training-Free Score Distillationcited by 3
  25. 2025Perspective-aware 3D Gaussian Inpainting with Multi-view Consistencycited by 3
  26. 2025Breaking the Vicious Cycle: Coherent 3D Gaussian Splatting from Sparse and Motion-Blurred Viewscited by 3
  27. 2026ShapeUP: Scalable Image-Conditioned 3D Editingcited by 3
  28. 2025HOMER: Homography-Based Efficient Multi-view 3D Object Removalcited by 2
  29. 2025Fillerbuster: Unified Generative Scene Completion Model for Casual Capturescited by 2
  30. 2025SplatFill: 3D Scene Inpainting via Depth-Guided Gaussian Splattingcited by 2
  31. 2026ArtisanGS: Interactive Tools for Gaussian Splat Selection with AI and Human in the Loopcited by 2
  32. 2024Semantic UV mapping to improve texture inpainting for indoor scenescited by 1
  33. 2024SR-CurvANN: Advancing 3D Surface Reconstruction through Curvature-Aware Neural Networkscited by 1
  34. 2024Towards High-Fidelity 3D Portrait Generation with Rich Details by Cross-View Prior-Aware Diffusioncited by 1
  35. 2024NeRF Inpainting with Geometric Diffusion Prior and Balanced Score Distillationcited by 1
  36. 2025ROODI: Reconstructing Occluded Objects with Denoising Inpainterscited by 1
  37. 2025DeclutterNeRF: Generative-Free 3D Scene Recovery for Occlusion Removalcited by 1
  38. 2025Improved 3D Scene Stylization via Text-Guided Generative Image Editing with Region-Based Controlcited by 1

50 most cited of 62 All 62 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023Inpaint3D: 3D Scene Content Generation using 2D Inpainting Diffusion
  2. 2023Inpaint4DNeRF: Promptable Spatio-Temporal NeRF Inpainting with Generative Diffusion Models
  3. 20243D-Consistent Image Inpainting with Diffusion Models
  4. 2024MVIP-NeRF: Multi-view 3D Inpainting on NeRF Scenes via Diffusion Prior
  5. 2024Invisible Stitch: Generating Smooth 3D Scenes with Depth Inpainting
  6. 2025SceneCompleter: Dense 3D Scene Completion for Generative Novel View Synthesis
  7. 2025OccludeNeRF: Geometric-aware 3D Scene Inpainting with Collaborative Score Distillation in NeRF
  8. 2022RenderDiffusion: Image Diffusion for 3D Reconstruction, Inpainting and Generation