mlsanityData to 7 Oct 2026

Efficient-NeRF2NeRF: Streamlining Text-Driven 3D Editing with Multiview Correspondence-Enhanced Diffusion Models

arXiv 2312.08563 ↗First posted 13 Dec 2023cs.CVcited by 16

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

The advancement of text-driven 3D content editing has been blessed by the progress from 2D generative diffusion models. However, a major obstacle hindering the widespread adoption of 3D content editing is its time-intensive processing. This challenge arises from the iterative and refining steps required to achieve consistent 3D outputs from 2D image-based generative models. Recent state-of-the-art methods typically require optimization time ranging from tens of minutes to several hours to edit a 3D scene using a single GPU. In this work, we propose that by incorporating correspondence regularization into diffusion models, the process of 3D editing can be significantly accelerated. This approach is inspired by the notion that the estimated samples during diffusion should be multiview-consistent during the diffusion generation process. By leveraging this multiview consistency, we can edit 3D content at a much faster speed. In most scenarios, our proposed technique brings a 10× speed-up compared to the baseline method and completes the editing of a 3D scene in 2 minutes with comparable quality.

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 85 of its 98 references that are in the map. First posted on arXiv 13 Dec 2023.

Builds on

85 of 98 references.

Closest ancestors · 25

Same direction, newest first

  1. 2023GaussianEditor: Editing 3D Gaussians Delicately with Text Instructions27 Nov
  2. 2023Consistent-1-to-3: Consistent Image to 3D View Synthesis via Geometry-aware Diffusion Models4 Oct
  3. 2023Consistent123: One Image to Highly Consistent 3D Asset Using Case-Aware Diffusion Priors29 Sep
  4. 2023SyncDreamer: Generating Multiview-consistent Images from a Single-view Image7 Sep
  5. 2023MVDream: Multi-view Diffusion for 3D Generation31 Aug
  6. 2023IT3D: Improved Text-to-3D Generation with Explicit View Synthesis22 Aug
  7. 2023HD-Fusion: Detailed Text-to-3D Generation Leveraging Multiple Noise Estimation30 Jul
  8. 2023Points-to-3D: Bridging the Gap between Sparse Points and Shape-Controllable Text-to-3D Generation26 Jul
  9. 2023Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priors30 Jun
  10. 2023One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimization29 Jun
  11. 2023DreamTime: An Improved Optimization Strategy for Diffusion-Guided 3D Generation21 Jun
  12. 2023Viewset Diffusion: (0-)Image-Conditioned 3D Generative Models from 2D Data13 Jun
13 more
  1. 2023AvatarStudio: Text-driven Editing of 3D Dynamic Human Head Avatars1 Jun
  2. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation25 May
  3. 2023TextMesh: Generation of Realistic 3D Meshes From Text Prompts24 Apr
  4. 2023Make-It-3D: High-Fidelity 3D Creation from A Single Image with Diffusion Prior24 Mar
  5. 2023Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creation24 Mar
  6. 2023Vox-E: Text-guided Voxel Editing of 3D Objects21 Mar
  7. 2023Zero-1-to-3: Zero-shot One Image to 3D Object20 Mar
  8. 2023Let 2D Diffusion Model Know 3D-Consistency for Robust Text-to-3D Generation14 Mar
  9. 2022Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generation1 Dec
  10. 2022NeuralLift-360: Lifting An In-the-wild 2D Photo to A 3D Object with 360deg Views29 Nov
  11. 2022Magic3D: High-Resolution Text-to-3D Content Creation18 Nov
  12. 2022Novel View Synthesis with Diffusion Models6 Oct
  13. 2022DreamFusion: Text-to-3D using 2D Diffusion29 Sep

Other references · 60

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2020Denoising Diffusion Implicit Modelscited by 9,655
  5. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  6. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  7. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  8. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  9. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  10. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
  11. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  12. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,698
38 more
  1. 2016Perceptual Losses for Real-Time Style Transfer and Super-Resolutioncited by 2,678
  2. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,591
  3. 2017Arbitrary Style Transfer in Real-time with Adaptive Instance Normalizationcited by 1,998
  4. 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,903
  5. 2021Plenoxels: Radiance Fields without Neural Networkscited by 1,607
  6. 2022TensoRF: Tensorial Radiance Fieldscited by 1,275
  7. 2022Diffusion Models in Vision: A Surveycited by 1,008
  8. 2021Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstructioncited by 961
  9. 2021PlenOctrees for Real-time Rendering of Neural Radiance Fieldscited by 904
  10. 2023DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creationcited by 837
  11. 2022Language-driven Semantic Segmentationcited by 711
  12. 2023Nerfstudio: A Modular Framework for Neural Radiance Field Developmentcited by 596
  13. 2021Zero-Shot Text-Guided Object Generation with Dream Fieldscited by 513
  14. 2022Point-NeRF: Point-based Neural Radiance Fieldscited by 503
  15. 2020NeRV: Neural Reflectance and Visibility Fields for Relighting and View Synthesiscited by 493
  16. 2023Instruct-NeRF2NeRF: Editing 3D Scenes with Instructionscited by 482
  17. 2020NeRD: Neural Reflectance Decomposition from Image Collectionscited by 446
  18. 2022Decomposing NeRF for Editing via Feature Field Distillationcited by 370
  19. 2021CLIP-NeRF: Text-and-Image Driven Manipulation of Neural Radiance Fieldscited by 369
  20. 2023RealFusion: 360deg Reconstruction of Any Object from a Single Imagecited by 305
  21. 2023Text-to-3D using Gaussian Splattingcited by 299
  22. 2023DreamBooth3D: Subject-Driven Text-to-3D Generationcited by 235
  23. 2021Editing Conditional Radiance Fieldscited by 222
  24. 2022NeRF-Editing: Geometry Editing of Neural Radiance Fieldscited by 216
  25. 2021NeX: Real-time View Synthesis with Neural Basis Expansioncited by 215
  26. 2023MVDiffusion: Enabling Holistic Multi-view Image Generation with Correspondence-Aware Diffusioncited by 210
  27. 2021Neural-PIL: Neural Pre-Integrated Lighting for Reflectance Decompositioncited by 204
  28. 2022Neural Feature Fusion Fields: 3D Distillation of Self-Supervised 2D Image Representationscited by 195
  29. 2022SinNeRF: Training Neural Radiance Fields on Complex Scenes from a Single Imagecited by 165
  30. 2023DreamEditor: Text-Driven 3D Scene Editing with Neural Fieldscited by 154
  31. 2021Editable Free-viewpoint Video Using a Layered Neural Representationcited by 148
  32. 2022ARF: Artistic Radiance Fieldscited by 147
  33. 2022StylizedNeRF: Consistent 3D Scene Stylization as Stylized NeRF via 2D-3D Mutual Learningcited by 140
  34. 2023Re-imagine the Negative Prompt Algorithm: Transform 2D Diffusion into 3D, alleviate Janus problem and Beyondcited by 139
  35. 2022NeRF-Art: Text-Driven Neural Radiance Fields Stylizationcited by 126
  36. 2021Stylizing 3D Scene via Implicit Representation and HyperNetworkcited by 103
  37. 2023Anything-3D: Towards Single-view Anything Reconstruction in the Wildcited by 91
  38. 2021Learning to Stylize Novel Viewscited by 76

50 most cited of 60 All 60 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 11; 2025: 4; 2026: 1.

16 papers.

Sort:

By area:

  1. 2024View-Consistent 3D Editing with Gaussian Splattingcited by 67
  2. 2024AutoVFX: Physically Realistic Video Editing from Natural Language Instructionscited by 27
  3. 2024GaussianVTON: 3D Human Virtual Try-ON via Multi-Stage Gaussian Splatting Editing with Image Promptingcited by 22
  4. 2024Consolidating Attention Features for Multi-view Image Editingcited by 19
  5. 2024ConsistDreamer: 3D-Consistent 2D Diffusion for High-Fidelity Scene Editingcited by 18
  6. 2024ProEdit: Simple Progression is All You Need for High-Quality 3D Scene Editingcited by 12
  7. 2025Fast Multi-view Consistent 3D Editing with Video Priorscited by 12
  8. 2024TrAME: Trajectory-Anchored Multi-View Editing for Text-Guided 3D Gaussian Splatting Manipulationcited by 10
  9. 2025V2Edit: Versatile Video Diffusion Editor for Videos and 3D Scenescited by 7
  10. 2024DragGaussian: Enabling Drag-style Manipulation on 3D Gaussian Representationcited by 6
  11. 2024SyncNoise: Geometrically Consistent Noise Prediction for Text-based 3D Scene Editingcited by 3
  12. 2024Preserving Identity with Variational Score for General-purpose 3D Editingcited by 2
4 more
  1. 2025Personalize Your Gaussian: Consistent 3D Scene Personalization from a Single Imagecited by 2
  2. 20253D-Consistent Multi-View Editing by Correspondence Guidancecited by 1
  3. 2024ArCSEM: Artistic Colorization of SEM Images via Gaussian Splattingcited by 0
  4. 2026GeM-NR: Geometry-Aware Multi-View Editing for Nonrigid Scene Changescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 20223DDesigner: Towards Photorealistic 3D Object Generation and Editing with Text-guided Diffusion Models
  2. 2025Coupled Diffusion Sampling for Training-Free Multi-View Image Editing
  3. 2024Text-to-3D Generation by 2D Editing
  4. 2023ED-NeRF: Efficient Text-Guided Editing of 3D Scene with Latent Space NeRF
  5. 20253D-LATTE: Latent Space 3D Editing from Textual Instructions
  6. 2024BoostDream: Efficient Refining for High-Quality Text-to-3D Generation from Multi-View Diffusion
  7. 2025DualNeRF: Text-Driven 3D Scene Editing via Dual-Field Representation
  8. 2024Towards a Training Free Approach for 3D Scene Editing