# Mesh-Guided Neural Implicit Field Editing

arXiv: https://arxiv.org/abs/2312.02157 · HTML: https://mlsanity.com/paper/2312.02157/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 43 references that are in the map.
- First posted on arXiv 4 Dec 2023.

## Builds on (most cited first, 20 of 20)

- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [DreamFusion: Text-to-3D using 2D Diffusion](https://mlsanity.com/paper/2209.14988/) (2022)
- [Nerfies: Deformable Neural Radiance Fields](https://mlsanity.com/paper/2011.12948/) (2020)
- [A Point Set Generation Network for 3D Object Reconstruction from a Single Image](https://mlsanity.com/paper/1612.00603/) (2016)
- [Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representations](https://mlsanity.com/paper/1906.01618/) (2019)
- [GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fields](https://mlsanity.com/paper/2011.12100/) (2020)
- [Implicit Geometric Regularization for Learning Shapes](https://mlsanity.com/paper/2002.10099/) (2020)
- [GRAF: Generative Radiance Fields for 3D-Aware Image Synthesis](https://mlsanity.com/paper/2007.02442/) (2020)
- [UNISURF: Unifying Neural Implicit Surfaces and Radiance Fields for Multi-View Reconstruction](https://mlsanity.com/paper/2104.10078/) (2021)
- [AtlasNet: A Papier-Mach'e Approach to Learning 3D Surface Generation](https://mlsanity.com/paper/1802.05384/) (2018)
- [Zero-Shot Text-Guided Object Generation with Dream Fields](https://mlsanity.com/paper/2112.01455/) (2021)
- [Instruct-NeRF2NeRF: Editing 3D Scenes with Instructions](https://mlsanity.com/paper/2303.12789/) (2023)
- [Putting NeRF on a Diet: Semantically Consistent Few-Shot View Synthesis](https://mlsanity.com/paper/2104.00677/) (2021)
- [Extracting Triangular 3D Models, Materials, and Lighting From Images](https://mlsanity.com/paper/2111.12503/) (2021)
- [MobileNeRF: Exploiting the Polygon Rasterization Pipeline for Efficient Neural Field Rendering on Mobile Architectures](https://mlsanity.com/paper/2208.00277/) (2022)
- [Editing Conditional Radiance Fields](https://mlsanity.com/paper/2105.06466/) (2021)
- [RigNeRF: Fully Controllable Neural 3D Portraits](https://mlsanity.com/paper/2206.06481/) (2022)
- [SINE: Semantic-driven Image-based NeRF Editing with Prior-guided Editing Field](https://mlsanity.com/paper/2303.13277/) (2023)
- [FDNeRF: Few-shot Dynamic Neural Radiance Fields for Face Reconstruction and Expression Editing](https://mlsanity.com/paper/2208.05751/) (2022)
- [Neural Parameterization for Dynamic Human Head Editing](https://mlsanity.com/paper/2207.00210/) (2022)

## Built on by (most cited first, 10 of 10)

- [TIP-Editor: An Accurate 3D Editor Following Both Text-Prompts And Image-Prompts](https://mlsanity.com/paper/2401.14828/) (2024)
- [View-Consistent 3D Editing with Gaussian Splatting](https://mlsanity.com/paper/2403.11868/) (2024)
- [SplatFlow: Multi-View Rectified Flow Model for 3D Gaussian Splatting Synthesis](https://mlsanity.com/paper/2411.16443/) (2024)
- [Mani-GS: Gaussian Splatting Manipulation with Triangular Mesh](https://mlsanity.com/paper/2405.17811/) (2024)
- [Generative Object Insertion in Gaussian Splatting with a Multi-View Diffusion Model](https://mlsanity.com/paper/2409.16938/) (2024)
- [4DGS-Craft: Consistent and Interactive 4D Gaussian Splatting Editing](https://mlsanity.com/paper/2510.01991/) (2025)
- [HEIR: Learning Graph-Based Motion Hierarchies](https://mlsanity.com/paper/2510.26786/) (2025)
- [CADMorph: Geometry-Driven Parametric CAD Editing via a Plan-Generate-Verify Loop](https://mlsanity.com/paper/2512.11480/) (2025)
- [PointGT: Simultaneous Geometry and Texture Editing for Point-Based Representations](https://mlsanity.com/paper/2609.03341/) (2026)
- [P-CORE: Self-Supervised Surface Consistency for Point-Based Neural Editing](https://mlsanity.com/paper/2609.03349/) (2026)

## Abstract (arXiv)

Neural implicit fields have emerged as a powerful 3D representation for reconstructing and rendering photo-realistic views, yet they possess limited editability. Conversely, explicit 3D representations, such as polygonal meshes, offer ease of editing but may not be as suitable for rendering high-quality novel views. To harness the strengths of both representations, we propose a new approach that employs a mesh as a guiding mechanism in editing the neural radiance field. We first introduce a differentiable method using marching tetrahedra for polygonal mesh extraction from the neural implicit field and then design a differentiable color extractor to assign colors obtained from the volume renderings to this extracted mesh. This differentiable colored mesh allows gradient back-propagation from the explicit mesh to the implicit fields, empowering users to easily manipulate the geometry and color of neural implicit fields. To enhance user control from coarse-grained to fine-grained levels, we introduce an octree-based structure into its optimization. This structure prioritizes the edited regions and the surface part, making our method achieve fine-grained edits to the neural implicit field and accommodate various user modifications, including object additions, component removals, specific area deformations, and adjustments to local and global colors. Through extensive experiments involving diverse scenes and editing operations, we have demonstrated the capabilities and effectiveness of our method. Our project page is: urlhttps://cassiepython.github.io/MNeuEdit/