mlsanityData to 7 Oct 2026

RePaint-NeRF: NeRF Editting via Semantic Masks and Diffusion Models

arXiv 2306.05668 ↗First posted 9 Jun 2023cs.CV cs.AI cs.GRcited by 19

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

The emergence of Neural Radiance Fields (NeRF) has promoted the development of synthesized high-fidelity views of the intricate real world. However, it is still a very demanding task to repaint the content in NeRF. In this paper, we propose a novel framework that can take RGB images as input and alter the 3D content in neural scenes. Our work leverages existing diffusion models to guide changes in the designated 3D content. Specifically, we semantically select the target object and a pre-trained diffusion model will guide the NeRF model to generate new 3D objects, which can improve the editability, diversity, and application range of NeRF. Experiment results show that our algorithm is effective for editing 3D objects in NeRF under different text prompts, including editing appearance, shape, and more. We validate our method on both real-world datasets and synthetic-world datasets for these editing tasks. Please visit https://starstesla.github.io/repaintnerf for a better view of our results.

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 30 of its 39 references that are in the map. First posted on arXiv 9 Jun 2023.

Builds on

30 of 39 references.

Closest ancestors · 2

Same direction, newest first

  1. 2022Magic3D: High-Resolution Text-to-3D Content Creation18 Nov
  2. 2022DreamFusion: Text-to-3D using 2D Diffusion29 Sep

Other references · 28

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  5. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  6. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
  7. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,698
  8. 2021Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fieldscited by 1,957
  9. 2021Plenoxels: Radiance Fields without Neural Networkscited by 1,607
  10. 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,484
  11. 2020D-NeRF: Neural Radiance Fields for Dynamic Scenescited by 1,459
  12. 2021Depth-supervised NeRF: Fewer Views and Faster Training for Freecited by 732
16 more
  1. 2022Block-NeRF: Scalable Large Scene Neural View Synthesiscited by 720
  2. 2022Language-driven Semantic Segmentationcited by 711
  3. 2019Local Light Field Fusion: Practical View Synthesis with Prescriptive Sampling Guidelinescited by 672
  4. 2021Zero-Shot Text-Guided Object Generation with Dream Fieldscited by 513
  5. 2022Latent-NeRF for Shape-Guided Generation of 3D Shapes and Texturescited by 506
  6. 2021In-Place Scene Labelling and Understanding with Implicit Scene Representationcited by 411
  7. 2022Decomposing NeRF for Editing via Feature Field Distillationcited by 370
  8. 2021CLIP-NeRF: Text-and-Image Driven Manipulation of Neural Radiance Fieldscited by 369
  9. 2022Panoptic Neural Fields: A Semantic Object-Aware Neural Scene Representationcited by 243
  10. 2021Learning Object-Compositional Neural Radiance Field for Editable Scene Renderingcited by 228
  11. 2021Editing Conditional Radiance Fieldscited by 222
  12. 2022NeRF-Editing: Geometry Editing of Neural Radiance Fieldscited by 216
  13. 2022Deforming Radiance Fields with Cagescited by 78
  14. 2022Removing Objects From Neural Radiance Fieldscited by 66
  15. 2019AADS: Augmented Autonomous Driving Simulation using Data-driven Algorithmscited by 39
  16. 2022LaTeRF: Label and Text Driven Object Radiance Fieldscited by 38

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 4; 2024: 9; 2025: 5; 2026: 1.

19 papers.

Sort:

By area:

  1. 2024DGE: Direct Gaussian 3D Editing by Consistent Multi-view Editingcited by 92
  2. 2024View-Consistent 3D Editing with Gaussian Splattingcited by 67
  3. 2023Alchemist: Parametric Control of Material Properties with Diffusion Modelscited by 38
  4. 2023SHAP-EDITOR: Instruction-guided Latent 3D Editing in Secondscited by 36
  5. 2024Coin3D: Controllable and Interactive 3D Assets Generation with Proxy-Guided Conditioningcited by 26
  6. 2024MVIP-NeRF: Multi-view 3D Inpainting on NeRF Scenes via Diffusion Priorcited by 24
  7. 2023Customize your NeRF: Adaptive Source Driven 3D Scene Editing via Local-Global Iterative Trainingcited by 20
  8. 2025Fast Multi-view Consistent 3D Editing with Video Priorscited by 12
  9. 2024InstantStyleGaussian: Efficient Art Style Transfer with 3D Gaussian Splattingcited by 11
  10. 2024TrAME: Trajectory-Anchored Multi-View Editing for Text-Guided 3D Gaussian Splatting Manipulationcited by 10
  11. 2023O2-Recon: Completing 3D Reconstruction of Occluded Objects in the Scene with a Pre-trained 2D Diffusion Modelcited by 8
  12. 2024GaussianForest: Hierarchical-Hybrid 3D Gaussian Splatting for Compressed Scene Modelingcited by 4
7 more
  1. 2025Stable Score Distillationcited by 3
  2. 2024Towards a Training Free Approach for 3D Scene Editingcited by 2
  3. 2025Personalize Your Gaussian: Consistent 3D Scene Personalization from a Single Imagecited by 2
  4. 2024TEDRA: Text-based Editing of Dynamic and Photoreal Actorscited by 1
  5. 2025Online 3D Gaussian Splatting Modeling with Novel View Selectioncited by 0
  6. 2025CADMorph: Geometry-Driven Parametric CAD Editing via a Plan-Generate-Verify Loopcited by 0
  7. 2026First Shape, Then Meaning: Efficient Geometry and Semantics Learning for Indoor Reconstructioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024DATENeRF: Depth-Aware Text-based Editing of NeRFs
  2. 2024Taming Latent Diffusion Model for Neural Radiance Field Inpainting
  3. 2023Learning a Diffusion Prior for NeRFs
  4. 2023SKED: Sketch-guided Text-based 3D Editing
  5. 2023Edit-DiffNeRF: Editing 3D Neural Radiance Fields using 2D Diffusion Model
  6. 2024ViCA-NeRF: View-Consistency-Aware 3D Editing of Neural Radiance Fields
  7. 2023ED-NeRF: Efficient Text-Guided Editing of 3D Scene with Latent Space NeRF
  8. 2023Inpaint4DNeRF: Promptable Spatio-Temporal NeRF Inpainting with Generative Diffusion Models