# Multi-robot autonomous 3D reconstruction using Gaussian splatting with Semantic guidance

arXiv: https://arxiv.org/abs/2412.02249 · HTML: https://mlsanity.com/paper/2412.02249/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 31 references that are in the map.
- First posted on arXiv 3 Dec 2024.

## Builds on (most cited first, 25 of 25)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [Detecting Twenty-thousand Classes using Image-level Supervision](https://mlsanity.com/paper/2201.02605/) (2022)
- [Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AI](https://mlsanity.com/paper/2109.08238/) (2021)
- [Gibson Env: Real-World Perception for Embodied Agents](https://mlsanity.com/paper/1808.10654/) (2018)
- [MonoSDF: Exploring Monocular Geometric Cues for Neural Implicit Surface Reconstruction](https://mlsanity.com/paper/2206.00665/) (2022)
- [Gaussian Splatting SLAM](https://mlsanity.com/paper/2312.06741/) (2023)
- [ConceptGraphs: Open-Vocabulary 3D Scene Graphs for Perception and Planning](https://mlsanity.com/paper/2309.16650/) (2023)
- [Language-Grounded Indoor 3D Semantic Segmentation in the Wild](https://mlsanity.com/paper/2204.07761/) (2022)
- [Gaussian-SLAM: Photo-realistic Dense SLAM with Gaussian Splatting](https://mlsanity.com/paper/2312.10070/) (2023)
- [OVIR-3D: Open-Vocabulary 3D Instance Retrieval Without Training on 3D Data](https://mlsanity.com/paper/2311.02873/) (2023)
- [NeurAR: Neural Uncertainty for Autonomous 3D Reconstruction with Implicit Neural Representations](https://mlsanity.com/paper/2207.10985/) (2022)
- [FUEL: Fast UAV Exploration using Incremental Frontier Structure and Hierarchical Planning](https://mlsanity.com/paper/2010.11561/) (2020)
- [FIESTA: Fast Incremental Euclidean Distance Fields for Online Motion Planning of Aerial Robots](https://mlsanity.com/paper/1903.02144/) (2019)
- [GS-Planner: A Gaussian-Splatting-based Planning Framework for Active High-Fidelity Reconstruction](https://mlsanity.com/paper/2405.10142/) (2024)
- [H2-Mapping: Real-time Dense Mapping Using Hierarchical Hybrid Representation](https://mlsanity.com/paper/2306.03207/) (2023)
- [FASTER: Fast and Safe Trajectory Planner for Flights in Unknown Environments](https://mlsanity.com/paper/1903.03558/) (2019)
- [Active Scene Understanding via Online Semantic Reconstruction](https://mlsanity.com/paper/1906.07409/) (2019)
- [ASH: A Modern Framework for Parallel Spatial Hashing in 3D Perception](https://mlsanity.com/paper/2110.00511/) (2021)
- [Efficient View Path Planning for Autonomous Implicit Reconstruction](https://mlsanity.com/paper/2209.13159/) (2022)
- [Object-Aware Guidance for Autonomous Scene Reconstruction](https://mlsanity.com/paper/1805.07794/) (2018)
- [Multi-Sensor Next-Best-View Planning as Matroid-Constrained Submodular Maximization](https://mlsanity.com/paper/2007.02084/) (2020)
- [Asynchronous Collaborative Autoscanning with Mode Switching for Multi-Robot Scene Reconstruction](https://mlsanity.com/paper/2210.04413/) (2022)
- [Autonomous Implicit Indoor Scene Reconstruction with Frontier Exploration](https://mlsanity.com/paper/2404.10218/) (2024)

## Built on by (most cited first, 1 of 1)

- [Uncertainty-driven 3D Gaussian Splatting Active Mapping via Anisotropic Visibility Field](https://mlsanity.com/paper/2605.30342/) (2026)

## Abstract (arXiv)

Implicit neural representations and 3D Gaussian splatting (3DGS) have shown great potential for scene reconstruction. Recent studies have expanded their applications in autonomous reconstruction through task assignment methods. However, these methods are mainly limited to single robot, and rapid reconstruction of large-scale scenes remains challenging. Additionally, task-driven planning based on surface uncertainty is prone to being trapped in local optima. To this end, we propose the first 3DGS-based centralized multi-robot autonomous 3D reconstruction framework. To further reduce time cost of task generation and improve reconstruction quality, we integrate online open-vocabulary semantic segmentation with surface uncertainty of 3DGS, focusing view sampling on regions with high instance uncertainty. Finally, we develop a multi-robot collaboration strategy with mode and task assignments improving reconstruction quality while ensuring planning efficiency. Our method demonstrates the highest reconstruction quality among all planning methods and superior planning efficiency compared to existing multi-robot methods. We deploy our method on multiple robots, and results show that it can effectively plan view paths and reconstruct scenes with high quality.