mlsanityData to 7 Oct 2026

Customize your NeRF: Adaptive Source Driven 3D Scene Editing via Local-Global Iterative Training

arXiv 2312.01663 ↗First posted 4 Dec 2023cs.CV cs.AIcited by 20

Not on the map. Why?

Abstract (arXiv)

In this paper, we target the adaptive source driven 3D scene editing task by proposing a CustomNeRF model that unifies a text description or a reference image as the editing prompt. However, obtaining desired editing results conformed with the editing prompt is nontrivial since there exist two significant challenges, including accurate editing of only foreground regions and multi-view consistency given a single-view reference image. To tackle the first challenge, we propose a Local-Global Iterative Editing (LGIE) training scheme that alternates between foreground region editing and full-image editing, aimed at foreground-only manipulation while preserving the background. For the second challenge, we also design a class-guided regularization that exploits class priors within the generation model to alleviate the inconsistency problem among different views in image-driven editing. Extensive experiments show that our CustomNeRF produces precise editing results under various real scenes for both text- and image-driven settings.

Key facts. Cited by 20 papers in the map (data through 7 Oct 2026). Builds on 45 of its 51 references that are in the map. First posted on arXiv 4 Dec 2023.

Builds on

45 of 51 references.

References · 45

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  5. 2023Segment Anythingcited by 8,158
  6. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  7. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  8. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  9. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  10. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,698
  11. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,286
  12. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
33 more
  1. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
  2. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  3. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,591
  4. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,409
  5. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,231
  6. 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,903
  7. 2021NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstructioncited by 1,701
  8. 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,376
  9. 2022Magic3D: High-Resolution Text-to-3D Content Creationcited by 1,285
  10. 2022Multi-Concept Customization of Text-to-Image Diffusioncited by 1,099
  11. 2021Blended Diffusion for Text-driven Editing of Natural Imagescited by 924
  12. 2019Local Light Field Fusion: Practical View Synthesis with Prescriptive Sampling Guidelinescited by 672
  13. 2023Muse: Text-To-Image Generation via Masked Generative Transformerscited by 640
  14. 2022Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generationcited by 597
  15. 2021Zero-Shot Text-Guided Object Generation with Dream Fieldscited by 513
  16. 2019BlendedMVS: A Large-scale Dataset for Generalized Multi-view Stereo Networkscited by 487
  17. 2023Instruct-NeRF2NeRF: Editing 3D Scenes with Instructionscited by 482
  18. 2021CLIP-NeRF: Text-and-Image Driven Manipulation of Neural Radiance Fieldscited by 369
  19. 2023TEXTure: Text-Guided Texturing of 3D Shapescited by 303
  20. 2021CLIP-Forge: Towards Zero-Shot Text-to-Shape Generationcited by 254
  21. 2023DreamBooth3D: Subject-Driven Text-to-3D Generationcited by 235
  22. 2021Editing Conditional Radiance Fieldscited by 222
  23. 2022Splicing ViT Features for Semantic Appearance Transfercited by 209
  24. 2021Neural Articulated Radiance Fieldcited by 196
  25. 2023DreamEditor: Text-Driven 3D Scene Editing with Neural Fieldscited by 154
  26. 2022Dream3D: Zero-Shot Text-to-3D Synthesis Using 3D Shape Prior and Text-to-Image Diffusion Modelscited by 151
  27. 2022NeuMesh: Learning Disentangled Neural Mesh-based Implicit Field for Geometry and Texture Editingcited by 141
  28. 2023Vox-E: Text-guided Voxel Editing of 3D Objectscited by 114
  29. 2021NeuTex: Neural Texture Mapping for Volumetric Neural Renderingcited by 98
  30. 2023SINE: Semantic-driven Image-based NeRF Editing with Prior-guided Editing Fieldcited by 93
  31. 2022Deforming Radiance Fields with Cagescited by 78
  32. 2022VolTeMorph: Realtime, Controllable and Generalisable Animation of Volumetric Representationscited by 36
  33. 2023RePaint-NeRF: NeRF Editting via Semantic Masks and Diffusion Modelscited by 19

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 8; 2025: 8; 2026: 3.

20 papers.

Sort:

By area:

  1. 2024TIP-Editor: An Accurate 3D Editor Following Both Text-Prompts And Image-Promptscited by 76
  2. 2024EditSplat: Multi-View Fusion and Attention-Guided Optimization for View-Consistent 3D Scene Editing with 3D Gaussian Splattingcited by 40
  3. 2024LLMs Meet Multimodal Generation and Editing: A Surveycited by 35
  4. 20243DSceneEditor: Controllable 3D Scene Editing with Gaussian Splattingcited by 20
  5. 2025InterGSEdit: Interactive 3D Gaussian Splatting Editing with 3D Geometry-Consistent Attention Priorcited by 19
  6. 2025PlanGen: Towards Unified Layout Planning and Image Generation in Auto-Regressive Vision Language Modelscited by 18
  7. 2024InfiniDreamer: Arbitrarily Long Human Motion Generation via Segment Score Distillationcited by 12
  8. 2024CTRL-D: Controllable Dynamic 3D Scene Editing with Personalized 2D Diffusioncited by 11
  9. 2024TrAME: Trajectory-Anchored Multi-View Editing for Text-Guided 3D Gaussian Splatting Manipulationcited by 10
  10. 2025Feedforward 3D Editing via Text-Steerable Image-to-3Dcited by 6
  11. 2026TAG-MoE: Task-Aware Gating for Unified Generative Mixture-of-Expertscited by 6
  12. 2025CoreEditor: Correspondence-constrained Diffusion for Consistent 3D Editingcited by 5
8 more
  1. 2023Learning Naturally Aggregated Appearance for Efficient 3D Editingcited by 4
  2. 2025Creating Your Editable 3D Photorealistic Avatar with Tetrahedron-constrained Gaussian Splattingcited by 3
  3. 2025GaussEdit: Adaptive 3D Scene Editing with Text and Image Promptscited by 1
  4. 2025Debiasing Diffusion Priors via 3D Attention for Consistent Gaussian Splattingcited by 1
  5. 2026Fake3DGS: A Benchmark for 3D Manipulation Detection in Neural Renderingcited by 1
  6. 2024Towards More Accurate Fake Detection on Images Generated from Advanced Generative and Neural Rendering Modelscited by 0
  7. 2025InStyle: Instant Appearance Stylization of 3D Shapescited by 0
  8. 2026T3HG-Editor: Text-driven 3D Human Garment Editing with Body Priors Embedded in SMPL-Xcited by 0