# SemSegMap- 3D Segment-Based Semantic Localization

arXiv: https://arxiv.org/abs/2107.14715 · HTML: https://mlsanity.com/paper/2107.14715/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Jul 2021.

## Built on by (most cited first, 10 of 10)

- [BoxGraph: Semantic Place Recognition and Pose Estimation from 3D LiDAR](https://mlsanity.com/paper/2206.15154/) (2022)
- [Convolutional Bayesian Kernel Inference for 3D Semantic Mapping](https://mlsanity.com/paper/2209.10663/) (2022)
- [ForestLPR: LiDAR Place Recognition in Forests Attentioning Multiple BEV Density Images](https://mlsanity.com/paper/2503.04475/) (2025)
- [That's My Point: Compact Object-centric LiDAR Pose Estimation for Large-scale Outdoor Localisation](https://mlsanity.com/paper/2403.04755/) (2024)
- [REGRACE: A Robust and Efficient Graph-based Re-localization Algorithm using Consistency Evaluation](https://mlsanity.com/paper/2503.03599/) (2025)
- [Superquadric Object Representation for Optimization-based Semantic SLAM](https://mlsanity.com/paper/2109.09627/) (2021)
- [Descriptellation: Deep Learned Constellation Descriptors](https://mlsanity.com/paper/2203.00567/) (2022)
- [QuadricsReg: Large-Scale Point Cloud Registration using Quadric Primitives](https://mlsanity.com/paper/2412.02998/) (2024)
- [Single-Shot Global Localization via Graph-Theoretic Correspondence Matching](https://mlsanity.com/paper/2306.03641/) (2023)
- [Introspection in Learned Semantic Scene Graph Localisation](https://mlsanity.com/paper/2510.07053/) (2025)

## Abstract (arXiv)

Localization is an essential task for mobile autonomous robotic systems that want to use pre-existing maps or create new ones in the context of SLAM. Today, many robotic platforms are equipped with high-accuracy 3D LiDAR sensors, which allow a geometric mapping, and cameras able to provide semantic cues of the environment. Segment-based mapping and localization have been applied with great success to 3D point-cloud data, while semantic understanding has been shown to improve localization performance in vision based systems. In this paper we combine both modalities in SemSegMap, extending SegMap into a segment based mapping framework able to also leverage color and semantic data from the environment to improve localization accuracy and robustness. In particular, we present new segmentation and descriptor extraction processes. The segmentation process benefits from additional distance information from color and semantic class consistency resulting in more repeatable segments and more overlap after re-visiting a place. For the descriptor, a tight fusion approach in a deep-learned descriptor extraction network is performed leading to a higher descriptiveness for landmark matching. We demonstrate the advantages of this fusion on multiple simulated and real-world datasets and compare its performance to various baselines. We show that we are able to find 50.9% more high-accuracy prior-less global localizations compared to SegMap on challenging datasets using very compact maps while also providing accurate full 6 DoF pose estimates in real-time.