# GaussCtrl: Multi-View Consistent Text-Driven 3D Gaussian Splatting Editing

arXiv: https://arxiv.org/abs/2403.08733 · HTML: https://mlsanity.com/paper/2403.08733/

## Key facts

- Cited by 115 papers in the map (data through 14 Oct 2026).
- Builds on 43 of its 50 references that are in the map.
- First posted on arXiv 13 Mar 2024.

## Builds on (most cited first, 30 of 43)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection](https://mlsanity.com/paper/2303.05499/) (2023)
- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [DreamFusion: Text-to-3D using 2D Diffusion](https://mlsanity.com/paper/2209.14988/) (2022)
- [InstructPix2Pix: Learning to Follow Image Editing Instructions](https://mlsanity.com/paper/2211.09800/) (2022)
- [Prompt-to-Prompt Image Editing with Cross Attention Control](https://mlsanity.com/paper/2208.01626/) (2022)
- [Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields](https://mlsanity.com/paper/2111.12077/) (2021)
- [DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection](https://mlsanity.com/paper/2203.03605/) (2022)
- [Null-text Inversion for Editing Real Images using Guided Diffusion Models](https://mlsanity.com/paper/2211.09794/) (2022)
- [Grounded Language-Image Pre-training](https://mlsanity.com/paper/2112.03857/) (2021)
- [MasaCtrl: Tuning-Free Mutual Self-Attention Control for Consistent Image Synthesis and Editing](https://mlsanity.com/paper/2304.08465/) (2023)
- [Language-driven Semantic Segmentation](https://mlsanity.com/paper/2201.03546/) (2022)
- [Nerfstudio: A Modular Framework for Neural Radiance Field Development](https://mlsanity.com/paper/2302.04264/) (2023)
- [DAB-DETR: Dynamic Anchor Boxes are Better Queries for DETR](https://mlsanity.com/paper/2201.12329/) (2022)
- [BlendedMVS: A Large-scale Dataset for Generalized Multi-view Stereo Networks](https://mlsanity.com/paper/1911.10127/) (2019)
- [DN-DETR: Accelerate DETR Training by Introducing Query DeNoising](https://mlsanity.com/paper/2203.01305/) (2022)
- [Instruct-NeRF2NeRF: Editing 3D Scenes with Instructions](https://mlsanity.com/paper/2303.12789/) (2023)
- [Understanding Diffusion Models: A Unified Perspective](https://mlsanity.com/paper/2208.11970/) (2022)

## Built on by (most cited first, 30 of 115)

- [Recent Advances in 3D Gaussian Splatting](https://mlsanity.com/paper/2403.11134/) (2024)
- [3D Gaussian Splatting: Survey, Technologies, Challenges, and Opportunities](https://mlsanity.com/paper/2407.17418/) (2024)
- [EditSplat: Multi-View Fusion and Attention-Guided Optimization for View-Consistent 3D Scene Editing with 3D Gaussian Splatting](https://mlsanity.com/paper/2412.11520/) (2024)
- [SplatFlow: Multi-View Rectified Flow Model for 3D Gaussian Splatting Synthesis](https://mlsanity.com/paper/2411.16443/) (2024)
- [GStex: Per-Primitive Texturing of 2D Gaussian Splatting for Decoupled Appearance and Geometry Modeling](https://mlsanity.com/paper/2409.12954/) (2024)
- [3DSceneEditor: Controllable 3D Scene Editing with Gaussian Splatting](https://mlsanity.com/paper/2412.01583/) (2024)
- [InterGSEdit: Interactive 3D Gaussian Splatting Editing with 3D Geometry-Consistent Attention Prior](https://mlsanity.com/paper/2507.04961/) (2025)
- [A Survey on 3D Gaussian Splatting Applications: Segmentation, Editing, and Generation](https://mlsanity.com/paper/2508.09977/) (2025)
- [LangSurf: Language-Embedded Surface Gaussians for 3D Scene Understanding](https://mlsanity.com/paper/2412.17635/) (2024)
- [Skyfall-GS: Synthesizing Immersive 3D Urban Scenes from Satellite Imagery](https://mlsanity.com/paper/2510.15869/) (2025)
- [CLIP-GS: CLIP-Informed Gaussian Splatting for View-Consistent 3D Indoor Semantic Understanding](https://mlsanity.com/paper/2404.14249/) (2024)
- [A Survey on 3D Human Avatar Modeling -- From Reconstruction to Generation](https://mlsanity.com/paper/2406.04253/) (2024)
- [GS-VTON: Controllable 3D Virtual Try-on with Gaussian Splatting](https://mlsanity.com/paper/2410.05259/) (2024)
- [Fast Multi-view Consistent 3D Editing with Video Priors](https://mlsanity.com/paper/2511.23172/) (2025)
- [Pro3D-Editor : A Progressive-Views Perspective for Consistent and Precise 3D Editing](https://mlsanity.com/paper/2506.00512/) (2025)
- [3D-LATTE: Latent Space 3D Editing from Textual Instructions](https://mlsanity.com/paper/2509.00269/) (2025)
- [TrAME: Trajectory-Anchored Multi-View Editing for Text-Guided 3D Gaussian Splatting Manipulation](https://mlsanity.com/paper/2407.02034/) (2024)
- [VTON 360: High-Fidelity Virtual Try-On from Any Viewing Direction](https://mlsanity.com/paper/2503.12165/) (2025)
- [AdLift: Lifting Adversarial Perturbations to Safeguard 3D Gaussian Splatting Assets Against Instruction-Driven Editing](https://mlsanity.com/paper/2512.07247/) (2025)
- [Zero-1-to-A: Zero-Shot One Image to Animatable Head Avatars Using Video Diffusion](https://mlsanity.com/paper/2503.15851/) (2025)
- [Tinker: Diffusion's Gift to 3D--Multi-View Consistent Editing From Sparse Inputs without Per-Scene Optimization](https://mlsanity.com/paper/2508.14811/) (2025)
- [3D Gaussian Editing with A Single Image](https://mlsanity.com/paper/2408.07540/) (2024)
- [Morpheus: Text-Driven 3D Gaussian Splat Shape and Color Stylization](https://mlsanity.com/paper/2503.02009/) (2025)
- [CLIPGaussian: Universal and Multimodal Style Transfer Based on Gaussian Splatting](https://mlsanity.com/paper/2505.22854/) (2025)
- [3DOT: Texture Transfer for 3DGS Objects from a Single Reference Image](https://mlsanity.com/paper/2503.18853/) (2025)
- [Let it Snow! Animating 3D Gaussian Scenes with Dynamic Weather Effects via Physics-Guided Score Distillation](https://mlsanity.com/paper/2504.05296/) (2025)
- [Digital Twin Generation from Visual Data: A Survey](https://mlsanity.com/paper/2504.13159/) (2025)
- [WonderFree: Enhancing Novel View Quality and Cross-View Consistency for 3D Scene Exploration](https://mlsanity.com/paper/2506.20590/) (2025)
- [SplatPainter: Interactive Authoring of 3D Gaussians from 2D Edits via Test-Time Training](https://mlsanity.com/paper/2512.05354/) (2025)
- [Feedforward 3D Editing via Text-Steerable Image-to-3D](https://mlsanity.com/paper/2512.13678/) (2025)

## Abstract (arXiv)

We propose GaussCtrl, a text-driven method to edit a 3D scene reconstructed by the 3D Gaussian Splatting (3DGS). Our method first renders a collection of images by using the 3DGS and edits them by using a pre-trained 2D diffusion model (ControlNet) based on the input prompt, which is then used to optimise the 3D model. Our key contribution is multi-view consistent editing, which enables editing all images together instead of iteratively editing one image while updating the 3D model as in previous works. It leads to faster editing as well as higher visual quality. This is achieved by the two terms: (a) depth-conditioned editing that enforces geometric consistency across multi-view images by leveraging naturally consistent depth maps. (b) attention-based latent code alignment that unifies the appearance of edited images by conditioning their editing to several reference views through self and cross-view attention between images' latent representations. Experiments demonstrate that our method achieves faster editing and better visual results than previous state-of-the-art methods.