# Mani-GS: Gaussian Splatting Manipulation with Triangular Mesh

arXiv: https://arxiv.org/abs/2405.17811 · HTML: https://mlsanity.com/paper/2405.17811/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 47 of its 61 references that are in the map.
- First posted on arXiv 28 May 2024.

## Builds on (most cited first, 30 of 47)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction](https://mlsanity.com/paper/2106.10689/) (2021)
- [4D Gaussian Splatting for Real-Time Dynamic Scene Rendering](https://mlsanity.com/paper/2310.08528/) (2023)
- [DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creation](https://mlsanity.com/paper/2309.16653/) (2023)
- [Deformable 3D Gaussians for High-Fidelity Monocular Dynamic Scene Reconstruction](https://mlsanity.com/paper/2309.13101/) (2023)
- [SuGaR: Surface-Aligned Gaussian Splatting for Efficient 3D Mesh Reconstruction and High-Quality Mesh Rendering](https://mlsanity.com/paper/2311.12775/) (2023)
- [Point-NeRF: Point-based Neural Radiance Fields](https://mlsanity.com/paper/2201.08845/) (2022)
- [PhysGaussian: Physics-Integrated 3D Gaussians for Generative Dynamics](https://mlsanity.com/paper/2311.12198/) (2023)
- [CLIP-NeRF: Text-and-Image Driven Manipulation of Neural Radiance Fields](https://mlsanity.com/paper/2112.05139/) (2021)
- [DrivingGaussian: Composite Gaussian Splatting for Surrounding Dynamic Autonomous Driving Scenes](https://mlsanity.com/paper/2312.07920/) (2023)
- [SC-GS: Sparse-Controlled Gaussian Splatting for Editable Dynamic Scenes](https://mlsanity.com/paper/2312.14937/) (2023)
- [MobileNeRF: Exploiting the Polygon Rasterization Pipeline for Efficient Neural Field Rendering on Mobile Architectures](https://mlsanity.com/paper/2208.00277/) (2022)
- [GS-IR: 3D Gaussian Splatting for Inverse Rendering](https://mlsanity.com/paper/2311.16473/) (2023)
- [Editing Conditional Radiance Fields](https://mlsanity.com/paper/2105.06466/) (2021)
- [BakedSDF: Meshing Neural SDFs for Real-Time View Synthesis](https://mlsanity.com/paper/2302.14859/) (2023)
- [NeRF-Editing: Geometry Editing of Neural Radiance Fields](https://mlsanity.com/paper/2205.04978/) (2022)
- [Gaussian-Flow: 4D Reconstruction with Dynamic 3D Gaussian Particle](https://mlsanity.com/paper/2312.03431/) (2023)
- [GaussianAvatar: Towards Realistic Human Avatar Modeling from a Single Video via Animatable 3D Gaussians](https://mlsanity.com/paper/2312.02134/) (2023)
- [VR-GS: A Physical Dynamics-Aware Interactive Gaussian Splatting System in Virtual Reality](https://mlsanity.com/paper/2401.16663/) (2024)
- [NeuMesh: Learning Disentangled Neural Mesh-based Implicit Field for Geometry and Texture Editing](https://mlsanity.com/paper/2207.11911/) (2022)
- [Drivable 3D Gaussian Avatars](https://mlsanity.com/paper/2311.08581/) (2023)
- [Structured Local Radiance Fields for Human Avatar Modeling](https://mlsanity.com/paper/2203.14478/) (2022)
- [FENeRF: Face Editing in Neural Radiance Fields](https://mlsanity.com/paper/2111.15490/) (2021)
- [Delicate Textured Mesh Recovery from NeRF via Adaptive Surface Refinement](https://mlsanity.com/paper/2303.02091/) (2023)
- [NeILF: Neural Incident Light Field for Physically-based Material Estimation](https://mlsanity.com/paper/2203.07182/) (2022)
- [NeuTex: Neural Texture Mapping for Volumetric Neural Rendering](https://mlsanity.com/paper/2103.00762/) (2021)
- [MoFaNeRF: Morphable Facial Neural Radiance Field](https://mlsanity.com/paper/2112.02308/) (2021)
- [SINE: Semantic-driven Image-based NeRF Editing with Prior-guided Editing Field](https://mlsanity.com/paper/2303.13277/) (2023)
- [GaMeS: Mesh-Based Adapting and Modification of Gaussian Splatting](https://mlsanity.com/paper/2402.01459/) (2024)

## Built on by (most cited first, 22 of 22)

- [IDOL: Instant Photorealistic 3D Human Creation from a Single Image](https://mlsanity.com/paper/2412.14963/) (2024)
- [FATE: Full-head Gaussian Avatar with Textural Editing from Monocular Video](https://mlsanity.com/paper/2411.15604/) (2024)
- [UW-GS: Distractor-Aware 3D Gaussian Splatting for Enhanced Underwater Scene Reconstruction](https://mlsanity.com/paper/2410.01517/) (2024)
- [LongSplat: Online Generalizable 3D Gaussian Splatting from Long Sequence Images](https://mlsanity.com/paper/2507.16144/) (2025)
- [MaGS: Reconstructing and Simulating Dynamic 3D Objects with Mesh-adsorbed Gaussian Splatting](https://mlsanity.com/paper/2406.01593/) (2024)
- [Radiance Meshes for Volumetric Reconstruction](https://mlsanity.com/paper/2512.04076/) (2025)
- [LMMs Meet Object-Centric Vision: Understanding, Segmentation, Editing and Generation](https://mlsanity.com/paper/2604.11789/) (2026)
- [FruitNinja: 3D Object Interior Texture Generation with Gaussian Splatting](https://mlsanity.com/paper/2411.12089/) (2024)
- [SIMART: Decomposing Monolithic Meshes into Sim-ready Articulated Assets via MLLM](https://mlsanity.com/paper/2603.23386/) (2026)
- [GeoTexDensifier: Geometry-Texture-Aware Densification for High-Quality Photorealistic 3D Gaussian Splatting](https://mlsanity.com/paper/2412.16809/) (2024)
- [4DGS-Craft: Consistent and Interactive 4D Gaussian Splatting Editing](https://mlsanity.com/paper/2510.01991/) (2025)
- [DynamicTree: Interactive Real Tree Animation via Sparse Voxel Spectrum](https://mlsanity.com/paper/2510.22213/) (2025)
- [Improving Multi-View Reconstruction via Texture-Guided Gaussian-Mesh Joint Optimization](https://mlsanity.com/paper/2511.03950/) (2025)
- [Complex-Valued 2D Gaussian Representation for Computer-Generated Holography](https://mlsanity.com/paper/2511.15022/) (2025)
- [Proxy-Free Gaussian Splats Deformation with Splat-Based Surface Estimation](https://mlsanity.com/paper/2511.19542/) (2025)
- [COREA: Coupled Relightable 3D Gaussians and SDFs for Efficient Normal Alignment](https://mlsanity.com/paper/2512.07107/) (2025)
- [PhysSkin: Real-Time and Generalizable Physics-Based Animation via Self-Supervised Neural Skinning](https://mlsanity.com/paper/2603.23194/) (2026)
- [RigPAPR: Rig-Based Animation of Static Neural Point Clouds from a Fixed-Viewpoint Video](https://mlsanity.com/paper/2606.06685/) (2026)
- [Mesh2GS: White-Box 3DGS Construction via Plenoptic Sampling](https://mlsanity.com/paper/2606.21898/) (2026)
- [DR-GS: Physically-Based Deformable and Relightable 2D Gaussians](https://mlsanity.com/paper/2606.29379/) (2026)
- [Gaussian Sculpting: End-to-End Controllable Surface Reconstruction via Field Optimization](https://mlsanity.com/paper/2608.10602/) (2026)
- [P-CORE: Self-Supervised Surface Consistency for Point-Based Neural Editing](https://mlsanity.com/paper/2609.03349/) (2026)

## Abstract (arXiv)

Neural 3D representations such as Neural Radiance Fields (NeRF), excel at producing photo-realistic rendering results but lack the flexibility for manipulation and editing which is crucial for content creation. Previous works have attempted to address this issue by deforming a NeRF in canonical space or manipulating the radiance field based on an explicit mesh. However, manipulating NeRF is not highly controllable and requires a long training and inference time. With the emergence of 3D Gaussian Splatting (3DGS), extremely high-fidelity novel view synthesis can be achieved using an explicit point-based 3D representation with much faster training and rendering speed. However, there is still a lack of effective means to manipulate 3DGS freely while maintaining rendering quality. In this work, we aim to tackle the challenge of achieving manipulable photo-realistic rendering. We propose to utilize a triangular mesh to manipulate 3DGS directly with self-adaptation. This approach reduces the need to design various algorithms for different types of Gaussian manipulation. By utilizing a triangle shape-aware Gaussian binding and adapting method, we can achieve 3DGS manipulation and preserve high-fidelity rendering after manipulation. Our approach is capable of handling large deformations, local manipulations, and soft body simulations while keeping high-quality rendering. Furthermore, we demonstrate that our method is also effective with inaccurate meshes extracted from 3DGS. Experiments conducted demonstrate the effectiveness of our method and its superiority over baseline approaches.