# Towards a Training Free Approach for 3D Scene Editing arXiv: https://arxiv.org/abs/2412.12766 ยท HTML: https://mlsanity.com/paper/2412.12766/ ## Key facts - Cited by 2 papers in the map (data through 14 Oct 2026). - Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. - Builds on 34 of its 46 references that are in the map. - First posted on arXiv 17 Dec 2024. ## Placed in - Diffusion models / [3D, point cloud and scene generation](https://mlsanity.com/diffusion/3d-point-cloud-and-scene-generation/) ## Builds on (most cited first, 30 of 34) - [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021) - [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020) - [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021) - [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023) - [GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models](https://mlsanity.com/paper/2112.10741/) (2021) - [Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection](https://mlsanity.com/paper/2303.05499/) (2023) - [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017) - [InstructPix2Pix: Learning to Follow Image Editing Instructions](https://mlsanity.com/paper/2211.09800/) (2022) - [Prompt-to-Prompt Image Editing with Cross Attention Control](https://mlsanity.com/paper/2208.01626/) (2022) - [Imagic: Text-Based Real Image Editing with Diffusion Models](https://mlsanity.com/paper/2210.09276/) (2022) - [The Replica Dataset: A Digital Replica of Indoor Spaces](https://mlsanity.com/paper/1906.05797/) (2019) - [MasaCtrl: Tuning-Free Mutual Self-Attention Control for Consistent Image Synthesis and Editing](https://mlsanity.com/paper/2304.08465/) (2023) - [Point-E: A System for Generating 3D Point Clouds from Complex Prompts](https://mlsanity.com/paper/2212.08751/) (2022) - [DiffEdit: Diffusion-based semantic image editing with mask guidance](https://mlsanity.com/paper/2210.11427/) (2022) - [Instruct-NeRF2NeRF: Editing 3D Scenes with Instructions](https://mlsanity.com/paper/2303.12789/) (2023) - [Shap-E: Generating Conditional 3D Implicit Functions](https://mlsanity.com/paper/2305.02463/) (2023) - [ConceptGraphs: Open-Vocabulary 3D Scene Graphs for Perception and Planning](https://mlsanity.com/paper/2309.16650/) (2023) - [OpenMask3D: Open-Vocabulary 3D Instance Segmentation](https://mlsanity.com/paper/2306.13631/) (2023) - [DreamEditor: Text-Driven 3D Scene Editing with Neural Fields](https://mlsanity.com/paper/2306.13455/) (2023) - [DreamAvatar: Text-and-Shape Guided 3D Human Avatar Generation via Diffusion Models](https://mlsanity.com/paper/2304.00916/) (2023) - [Training-Free Consistent Text-to-Image Generation](https://mlsanity.com/paper/2402.03286/) (2024) - [Vox-E: Text-guided Voxel Editing of 3D Objects](https://mlsanity.com/paper/2303.12048/) (2023) - [SINE: Semantic-driven Image-based NeRF Editing with Prior-guided Editing Field](https://mlsanity.com/paper/2303.13277/) (2023) - [Set-the-Scene: Global-Local Training for Generating Controllable NeRF Scenes](https://mlsanity.com/paper/2303.13450/) (2023) - [SKED: Sketch-guided Text-based 3D Editing](https://mlsanity.com/paper/2303.10735/) (2023) - [ViCA-NeRF: View-Consistency-Aware 3D Editing of Neural Radiance Fields](https://mlsanity.com/paper/2402.00864/) (2024) - [Advances in 3D Generation: A Survey](https://mlsanity.com/paper/2401.17807/) (2024) - [Control-NeRF: Editable Feature Volumes for Scene Rendering and Manipulation](https://mlsanity.com/paper/2204.10850/) (2022) - [Grasp'D: Differentiable Contact-rich Grasp Synthesis for Multi-fingered Hands](https://mlsanity.com/paper/2208.12250/) (2022) - [Instruct 3D-to-3D: Text Instruction Guided 3D-to-3D conversion](https://mlsanity.com/paper/2303.15780/) (2023) ## Built on by (most cited first, 2 of 2) - [Towards Scalable and Consistent 3D Editing](https://mlsanity.com/paper/2510.02994/) (2025) - [Edit3r: Instant 3D Scene Editing from Sparse Unposed Images](https://mlsanity.com/paper/2512.25071/) (2025) ## Abstract (arXiv) Text driven diffusion models have shown remarkable capabilities in editing images. However, when editing 3D scenes, existing works mostly rely on training a NeRF for 3D editing. Recent NeRF editing methods leverages edit operations by deploying 2D diffusion models and project these edits into 3D space. They require strong positional priors alongside text prompt to identify the edit location. These methods are operational on small 3D scenes and are more generalized to particular scene. They require training for each specific edit and cannot be exploited in real-time edits. To address these limitations, we propose a novel method, FreeEdit, to make edits in training free manner using mesh representations as a substitute for NeRF. Training-free methods are now a possibility because of the advances in foundation model's space. We leverage these models to bring a training-free alternative and introduce solutions for insertion, replacement and deletion. We consider insertion, replacement and deletion as basic blocks for performing intricate edits with certain combinations of these operations. Given a text prompt and a 3D scene, our model is capable of identifying what object should be inserted/replaced or deleted and location where edit should be performed. We also introduce a novel algorithm as part of FreeEdit to find the optimal location on grounding object for placement. We evaluate our model by comparing it with baseline models on a wide range of scenes using quantitative and qualitative metrics and showcase the merits of our method with respect to others.