# Kimera-Multi: Robust, Distributed, Dense Metric-Semantic SLAM for Multi-Robot Systems

arXiv: https://arxiv.org/abs/2106.14386 · HTML: https://mlsanity.com/paper/2106.14386/

## Key facts

- Cited by 54 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 102 references that are in the map.
- First posted on arXiv 28 Jun 2021.

## Builds on (most cited first, 30 of 36)

- [NetVLAD: CNN architecture for weakly supervised place recognition](https://mlsanity.com/paper/1511.07247/) (2015)
- [Open3D: A Modern Library for 3D Data Processing](https://mlsanity.com/paper/1801.09847/) (2018)
- [Kimera: an Open-Source Library for Real-Time Metric-Semantic Localization and Mapping](https://mlsanity.com/paper/1910.02490/) (2019)
- [CNN-SLAM: Real-time dense monocular SLAM with learned depth prediction](https://mlsanity.com/paper/1704.03489/) (2017)
- [3D Dynamic Scene Graphs: Actionable Spatial Perception with Places, Objects, and Humans](https://mlsanity.com/paper/2002.06289/) (2020)
- [Kimera: from SLAM to Spatial Perception with 3D Dynamic Scene Graphs](https://mlsanity.com/paper/2101.06894/) (2021)
- [SemanticFusion: Dense 3D Semantic Mapping with Convolutional Neural Networks](https://mlsanity.com/paper/1609.05130/) (2016)
- [A Review on Deep Learning Techniques Applied to Semantic Segmentation](https://mlsanity.com/paper/1704.06857/) (2017)
- [Fusion++: Volumetric Object-Level SLAM](https://mlsanity.com/paper/1808.08378/) (2018)
- [On-Manifold Preintegration for Real-Time Visual-Inertial Odometry](https://mlsanity.com/paper/1512.02363/) (2015)
- [MaskFusion: Real-Time Recognition, Tracking and Reconstruction of Multiple Moving Objects](https://mlsanity.com/paper/1804.09194/) (2018)
- [Graduated Non-Convexity for Robust Spatial Perception: From Non-Minimal Solvers to Global Outlier Rejection](https://mlsanity.com/paper/1909.08605/) (2019)
- [SE-Sync: A Certifiably Correct Algorithm for Synchronization over the Special Euclidean Group](https://mlsanity.com/paper/1612.07386/) (2016)
- [Volumetric Instance-Aware Semantic Mapping and 3D Object Discovery](https://mlsanity.com/paper/1903.00268/) (2019)
- [MID-Fusion: Octree-based Object-Level Multi-Instance Dynamic SLAM](https://mlsanity.com/paper/1812.07976/) (2018)
- [Co-Fusion: Real-time Segmentation, Tracking and Fusion of Multiple Objects](https://mlsanity.com/paper/1706.06629/) (2017)
- [FutureMapping: The Computational Structure of Spatial AI Systems](https://mlsanity.com/paper/1803.11288/) (2018)
- [NeBula: Quest for Robotic Autonomy in Challenging Environments; TEAM CoSTAR at the DARPA Subterranean Challenge](https://mlsanity.com/paper/2103.11470/) (2021)
- [DOOR-SLAM: Distributed, Online, and Outlier Resilient SLAM for Robotic Teams](https://mlsanity.com/paper/1909.12198/) (2019)
- [Outlier-Robust Estimation: Hardness, Minimally Tuned Algorithms, and Applications](https://mlsanity.com/paper/2007.15109/) (2020)
- [Modeling Perceptual Aliasing in SLAM via Discrete-Continuous Graphical Models](https://mlsanity.com/paper/1810.11692/) (2018)
- [Data-Efficient Decentralized Visual SLAM](https://mlsanity.com/paper/1710.05772/) (2017)
- [Convex Relaxations for Pose Graph Optimization with Outliers](https://mlsanity.com/paper/1801.02112/) (2018)
- [Kimera-Multi: a System for Distributed Multi-Robot Metric-Semantic Simultaneous Localization and Mapping](https://mlsanity.com/paper/2011.04087/) (2020)
- [Distributed Certifiably Correct Pose-Graph Optimization](https://mlsanity.com/paper/1911.03721/) (2019)
- [Parallel Maximum Clique Algorithms with Applications to Network Analysis and Storage](https://mlsanity.com/paper/1302.6256/) (2013)
- [Distributed Mapping with Privacy and Communication Constraints: Lightweight Algorithms and Object-based Models](https://mlsanity.com/paper/1702.03435/) (2017)
- [Incremental Visual-Inertial 3D Mesh Generation with Structural Regularities](https://mlsanity.com/paper/1903.01067/) (2019)
- [Active Scene Understanding via Online Semantic Reconstruction](https://mlsanity.com/paper/1906.07409/) (2019)
- [Efficient Decentralized Visual Place Recognition From Full-Image Descriptors](https://mlsanity.com/paper/1705.10739/) (2017)

## Built on by (most cited first, 30 of 54)

- [SemGauss-SLAM: Dense Semantic Gaussian Splatting SLAM](https://mlsanity.com/paper/2403.07494/) (2024)
- [Learning Generalizable Feature Fields for Mobile Manipulation](https://mlsanity.com/paper/2403.07563/) (2024)
- [General Place Recognition Survey: Towards Real-World Autonomy](https://mlsanity.com/paper/2405.04812/) (2024)
- [Swarm-SLAM : Sparse Decentralized Collaborative Simultaneous Localization and Mapping Framework for Multi-Robot Systems](https://mlsanity.com/paper/2301.06230/) (2023)
- [From SLAM to Situational Awareness: Challenges and Survey](https://mlsanity.com/paper/2110.00273/) (2021)
- [BioSLAM: A Bio-inspired Lifelong Memory System for General Place Recognition](https://mlsanity.com/paper/2208.14543/) (2022)
- [Beyond Robustness: A Taxonomy of Approaches towards Resilient Multi-Robot Systems](https://mlsanity.com/paper/2109.12343/) (2021)
- [General Place Recognition Survey: Towards the Real-world Autonomy Age](https://mlsanity.com/paper/2209.04497/) (2022)
- [Air-Ground Collaboration for Language-Specified Missions in Unknown Environments](https://mlsanity.com/paper/2505.09108/) (2025)
- [MCN-SLAM: Multi-Agent Collaborative Neural SLAM with Hybrid Implicit Neural Scene Representation](https://mlsanity.com/paper/2506.18678/) (2025)
- [AutoMerge: A Framework for Map Assembling and Smoothing in City-scale Environments](https://mlsanity.com/paper/2207.06965/) (2022)
- [SG-Reg: Generalizable and Efficient Scene Graph Registration](https://mlsanity.com/paper/2504.14440/) (2025)
- [Recent Advancements in Deep Learning Applications and Methods for Autonomous Navigation: A Comprehensive Review](https://mlsanity.com/paper/2302.11089/) (2023)
- [MAC-Ego3D: Multi-Agent Gaussian Consensus for Real-Time Collaborative Ego-Motion and Photorealistic 3D Reconstruction](https://mlsanity.com/paper/2412.09723/) (2024)
- [LiDAR-VGGT: Cross-Modal Coarse-to-Fine Fusion for Globally Consistent and Metric-Scale Dense Mapping](https://mlsanity.com/paper/2511.01186/) (2025)
- [SOS-Match: Segmentation for Open-Set Robust Correspondence Search and Robot Localization in Unstructured Environments](https://mlsanity.com/paper/2401.04791/) (2024)
- [Environment-Driven Online LiDAR-Camera Extrinsic Calibration](https://mlsanity.com/paper/2502.00801/) (2025)
- [Heterogeneous Robot Collaboration in Unstructured Environments with Grounded Generative Intelligence](https://mlsanity.com/paper/2510.26915/) (2025)
- [Real-World Deployment of a Hierarchical Uncertainty-Aware Collaborative Multiagent Planning System](https://mlsanity.com/paper/2404.17438/) (2024)
- [Communication- and Computation-Efficient Distributed Submodular Optimization in Robot Mesh Networks](https://mlsanity.com/paper/2407.10382/) (2024)
- [OpenObject-NAV: Open-Vocabulary Object-Oriented Navigation Based on Dynamic Carrier-Relationship Scene Graph](https://mlsanity.com/paper/2409.18743/) (2024)
- [Real-Time Metric-Semantic Mapping for Autonomous Navigation in Outdoor Environments](https://mlsanity.com/paper/2412.00291/) (2024)
- [MIXER: Multiattribute, Multiway Fusion of Uncertain Pairwise Affinities](https://mlsanity.com/paper/2210.08360/) (2022)
- [SeMLaPS: Real-time Semantic Mapping with Latent Prior Networks and Quasi-Planar Segmentation](https://mlsanity.com/paper/2306.16585/) (2023)
- [Outlier-Robust Long-Term Robotic Mapping Leveraging Ground Segmentation](https://mlsanity.com/paper/2405.11176/) (2024)
- [Efficient and Distributed Large-Scale 3D Map Registration using Tomographic Features](https://mlsanity.com/paper/2406.19461/) (2024)
- [SPACE: 3D Spatial Co-operation and Exploration Framework for Robust Mapping and Coverage with Multi-Robot Systems](https://mlsanity.com/paper/2411.02524/) (2024)
- [Policies over Poses: Reinforcement Learning based Distributed Pose-Graph Optimization for Multi-Robot SLAM](https://mlsanity.com/paper/2510.22740/) (2025)
- [DST-Calib: A Dual-Path, Self-Supervised, Target-Free LiDAR-Camera Extrinsic Calibration Network](https://mlsanity.com/paper/2601.01188/) (2026)
- [Dense Dynamic Scene Reconstruction and Camera Pose Estimation from Multi-View Videos](https://mlsanity.com/paper/2603.12064/) (2026)

## Abstract (arXiv)

This paper presents Kimera-Multi, the first multi-robot system that (i) is robust and capable of identifying and rejecting incorrect inter and intra-robot loop closures resulting from perceptual aliasing, (ii) is fully distributed and only relies on local (peer-to-peer) communication to achieve distributed localization and mapping, and (iii) builds a globally consistent metric-semantic 3D mesh model of the environment in real-time, where faces of the mesh are annotated with semantic labels. Kimera-Multi is implemented by a team of robots equipped with visual-inertial sensors. Each robot builds a local trajectory estimate and a local mesh using Kimera. When communication is available, robots initiate a distributed place recognition and robust pose graph optimization protocol based on a novel distributed graduated non-convexity algorithm. The proposed protocol allows the robots to improve their local trajectory estimates by leveraging inter-robot loop closures while being robust to outliers. Finally, each robot uses its improved trajectory estimate to correct the local mesh using mesh deformation techniques. We demonstrate Kimera-Multi in photo-realistic simulations, SLAM benchmarking datasets, and challenging outdoor datasets collected using ground robots. Both real and simulated experiments involve long trajectories (e.g., up to 800 meters per robot). The experiments show that Kimera-Multi (i) outperforms the state of the art in terms of robustness and accuracy, (ii) achieves estimation errors comparable to a centralized SLAM system while being fully distributed, (iii) is parsimonious in terms of communication bandwidth, (iv) produces accurate metric-semantic 3D meshes, and (v) is modular and can be also used for standard 3D reconstruction (i.e., without semantic labels) or for trajectory estimation (i.e., without reconstructing a 3D mesh).