# ViCA-NeRF: View-Consistency-Aware 3D Editing of Neural Radiance Fields

arXiv: https://arxiv.org/abs/2402.00864 · HTML: https://mlsanity.com/paper/2402.00864/

## Key facts

- Cited by 68 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 37 references that are in the map.
- First posted on arXiv 1 Feb 2024.

## Builds on (most cited first, 30 of 33)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [DreamFusion: Text-to-3D using 2D Diffusion](https://mlsanity.com/paper/2209.14988/) (2022)
- [InstructPix2Pix: Learning to Follow Image Editing Instructions](https://mlsanity.com/paper/2211.09800/) (2022)
- [Prompt-to-Prompt Image Editing with Cross Attention Control](https://mlsanity.com/paper/2208.01626/) (2022)
- [SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations](https://mlsanity.com/paper/2108.01073/) (2021)
- [D-NeRF: Neural Radiance Fields for Dynamic Scenes](https://mlsanity.com/paper/2011.13961/) (2020)
- [Magic3D: High-Resolution Text-to-3D Content Creation](https://mlsanity.com/paper/2211.10440/) (2022)
- [NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections](https://mlsanity.com/paper/2008.02268/) (2020)
- [Imagic: Text-Based Real Image Editing with Diffusion Models](https://mlsanity.com/paper/2210.09276/) (2022)
- [Blended Diffusion for Text-driven Editing of Natural Images](https://mlsanity.com/paper/2111.14818/) (2021)
- [PlenOctrees for Real-time Rendering of Neural Radiance Fields](https://mlsanity.com/paper/2103.14024/) (2021)
- [Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creation](https://mlsanity.com/paper/2303.13873/) (2023)
- [Nerfstudio: A Modular Framework for Neural Radiance Field Development](https://mlsanity.com/paper/2302.04264/) (2023)
- [Instruct-NeRF2NeRF: Editing 3D Scenes with Instructions](https://mlsanity.com/paper/2303.12789/) (2023)
- [Decomposing NeRF for Editing via Feature Field Distillation](https://mlsanity.com/paper/2205.15585/) (2022)
- [Text2Mesh: Text-Driven Neural Stylization for Meshes](https://mlsanity.com/paper/2112.03221/) (2021)
- [DreamBooth3D: Subject-Driven Text-to-3D Generation](https://mlsanity.com/paper/2303.13508/) (2023)
- [Learning Object-Compositional Neural Radiance Field for Editable Scene Rendering](https://mlsanity.com/paper/2109.01847/) (2021)
- [Editing Conditional Radiance Fields](https://mlsanity.com/paper/2105.06466/) (2021)
- [StylizedNeRF: Consistent 3D Scene Stylization as Stylized NeRF via 2D-3D Mutual Learning](https://mlsanity.com/paper/2205.12183/) (2022)
- [NeRF-Art: Text-Driven Neural Radiance Fields Stylization](https://mlsanity.com/paper/2212.08070/) (2022)
- [Stylizing 3D Scene via Implicit Representation and HyperNetwork](https://mlsanity.com/paper/2105.13016/) (2021)
- [SINE: Semantic-driven Image-based NeRF Editing with Prior-guided Editing Field](https://mlsanity.com/paper/2303.13277/) (2023)
- [Neural 3D Reconstruction in the Wild](https://mlsanity.com/paper/2205.12955/) (2022)
- [Learning to Stylize Novel Views](https://mlsanity.com/paper/2105.13509/) (2021)

## Built on by (most cited first, 30 of 68)

- [GaussCtrl: Multi-View Consistent Text-Driven 3D Gaussian Splatting Editing](https://mlsanity.com/paper/2403.08733/) (2024)
- [DGE: Direct Gaussian 3D Editing by Consistent Multi-view Editing](https://mlsanity.com/paper/2404.18929/) (2024)
- [View-Consistent 3D Editing with Gaussian Splatting](https://mlsanity.com/paper/2403.11868/) (2024)
- [EditSplat: Multi-View Fusion and Attention-Guided Optimization for View-Consistent 3D Scene Editing with 3D Gaussian Splatting](https://mlsanity.com/paper/2412.11520/) (2024)
- [GeoDiffuser: Geometry-Based Image Editing with Diffusion Models](https://mlsanity.com/paper/2404.14403/) (2024)
- [AutoVFX: Physically Realistic Video Editing from Natural Language Instructions](https://mlsanity.com/paper/2411.02394/) (2024)
- [Instruct 4D-to-4D: Editing 4D Scenes as Pseudo-3D Scenes Using 2D Diffusion](https://mlsanity.com/paper/2406.09402/) (2024)
- [GaussianVTON: 3D Human Virtual Try-ON via Multi-Stage Gaussian Splatting Editing with Image Prompting](https://mlsanity.com/paper/2405.07472/) (2024)
- [3DSceneEditor: Controllable 3D Scene Editing with Gaussian Splatting](https://mlsanity.com/paper/2412.01583/) (2024)
- [ConsistDreamer: 3D-Consistent 2D Diffusion for High-Fidelity Scene Editing](https://mlsanity.com/paper/2406.09404/) (2024)
- [Style-NeRF2NeRF: 3D Style Transfer From Style-Aligned Multi-View Images](https://mlsanity.com/paper/2406.13393/) (2024)
- [Customizing Text-to-Image Diffusion with Object Viewpoint Control](https://mlsanity.com/paper/2404.12333/) (2024)
- [Localized Gaussian Splatting Editing with Contextual Awareness](https://mlsanity.com/paper/2408.00083/) (2024)
- [GS-VTON: Controllable 3D Virtual Try-on with Gaussian Splatting](https://mlsanity.com/paper/2410.05259/) (2024)
- [ProEdit: Simple Progression is All You Need for High-Quality 3D Scene Editing](https://mlsanity.com/paper/2411.05006/) (2024)
- [Fast Multi-view Consistent 3D Editing with Video Priors](https://mlsanity.com/paper/2511.23172/) (2025)
- [TrAME: Trajectory-Anchored Multi-View Editing for Text-Guided 3D Gaussian Splatting Manipulation](https://mlsanity.com/paper/2407.02034/) (2024)
- [3DEgo: 3D Editing on the Go!](https://mlsanity.com/paper/2407.10102/) (2024)
- [CL-Splats: Continual Learning of Gaussian Splatting with Local Optimization](https://mlsanity.com/paper/2506.21117/) (2025)
- [TexGS-VolVis: Expressive Scene Editing for Volume Visualization via Textured Gaussian Splatting](https://mlsanity.com/paper/2507.13586/) (2025)
- [Morpheus: Text-Driven 3D Gaussian Splat Shape and Color Stylization](https://mlsanity.com/paper/2503.02009/) (2025)
- [Generative Object Insertion in Gaussian Splatting with a Multi-View Diffusion Model](https://mlsanity.com/paper/2409.16938/) (2024)
- [Geometry-Aware Diffusion Models for Multiview Scene Inpainting](https://mlsanity.com/paper/2502.13335/) (2025)
- [Omni-3DEdit: Generalized Versatile 3D Editing in One-Pass](https://mlsanity.com/paper/2603.17841/) (2026)
- [DATENeRF: Depth-Aware Text-based Editing of NeRFs](https://mlsanity.com/paper/2404.04526/) (2024)
- [FreeInsert: Disentangled Text-Guided Object Insertion in 3D Gaussian Scene without Spatial Priors](https://mlsanity.com/paper/2505.01322/) (2025)
- [Robust 3D-Masked Part-level Editing in 3D Gaussian Splatting with Regularized Score Distillation Sampling](https://mlsanity.com/paper/2507.11061/) (2025)
- [CoreEditor: Correspondence-constrained Diffusion for Consistent 3D Editing](https://mlsanity.com/paper/2508.11603/) (2025)
- [C3Editor: Achieving Controllable Consistency in 2D Model for 3D Editing](https://mlsanity.com/paper/2510.04539/) (2025)
- [ArtNVG: Content-Style Separated Artistic Neighboring-View Gaussian Stylization](https://mlsanity.com/paper/2412.18783/) (2024)

## Abstract (arXiv)

We introduce ViCA-NeRF, the first view-consistency-aware method for 3D editing with text instructions. In addition to the implicit neural radiance field (NeRF) modeling, our key insight is to exploit two sources of regularization that explicitly propagate the editing information across different views, thus ensuring multi-view consistency. For geometric regularization, we leverage the depth information derived from NeRF to establish image correspondences between different views. For learned regularization, we align the latent codes in the 2D diffusion model between edited and unedited images, enabling us to edit key views and propagate the update throughout the entire scene. Incorporating these two strategies, our ViCA-NeRF operates in two stages. In the initial stage, we blend edits from different views to create a preliminary 3D edit. This is followed by a second stage of NeRF training, dedicated to further refining the scene's appearance. Experimental results demonstrate that ViCA-NeRF provides more flexible, efficient (3 times faster) editing with higher levels of consistency and details, compared with the state of the art. Our code is publicly available.