# Long-term Visual Localization using Semantically Segmented Images

arXiv: https://arxiv.org/abs/1801.05269 · HTML: https://mlsanity.com/paper/1801.05269/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Jan 2018.

## Built on by (most cited first, 15 of 15)

- [SFD2: Semantic-guided Feature Detection and Description](https://mlsanity.com/paper/2304.14845/) (2023)
- [CVLNet: Cross-View Semantic Correspondence Learning for Video-based Camera Localization](https://mlsanity.com/paper/2208.03660/) (2022)
- [RTG-SLAM: Real-time 3D Reconstruction at Scale using Gaussian Splatting](https://mlsanity.com/paper/2404.19706/) (2024)
- [Semantic Visual Simultaneous Localization and Mapping: A Survey](https://mlsanity.com/paper/2209.06428/) (2022)
- [Semantic Pose Verification for Outdoor Visual Localization with Self-supervised Contrastive Learning](https://mlsanity.com/paper/2203.16945/) (2022)
- [S-BEV: Semantic Birds-Eye View Representation for Weather and Lighting Invariant 3-DoF Localization](https://mlsanity.com/paper/2101.09569/) (2021)
- [View-Invariant Localization using Semantic Objects in Changing Environments](https://mlsanity.com/paper/2209.14426/) (2022)
- [Context-Based Visual-Language Place Recognition](https://mlsanity.com/paper/2410.19341/) (2024)
- [Semantic Image Alignment for Vehicle Localization](https://mlsanity.com/paper/2110.04162/) (2021)
- [Training Semantic Descriptors for Image-Based Localization](https://mlsanity.com/paper/2202.01212/) (2022)
- [Monocular Localization with Semantics Map for Autonomous Vehicles](https://mlsanity.com/paper/2406.03835/) (2024)
- [Locking On: Leveraging Dynamic Vehicle-Imposed Motion Constraints to Improve Visual Localization](https://mlsanity.com/paper/2306.17529/) (2023)
- [PRAM: Place Recognition Anywhere Model for Efficient Visual Localization](https://mlsanity.com/paper/2404.07785/) (2024)
- [Map-Free Visual Relocalization Enhanced by Instance Knowledge and Depth Knowledge](https://mlsanity.com/paper/2408.13085/) (2024)
- [Semantic and Feature Guided Uncertainty Quantification of Visual Localization for Autonomous Vehicles](https://mlsanity.com/paper/2506.15851/) (2025)

## Abstract (arXiv)

Robust cross-seasonal localization is one of the major challenges in long-term visual navigation of autonomous vehicles. In this paper, we exploit recent advances in semantic segmentation of images, i.e., where each pixel is assigned a label related to the type of object it represents, to attack the problem of long-term visual localization. We show that semantically labeled 3-D point maps of the environment, together with semantically segmented images, can be efficiently used for vehicle localization without the need for detailed feature descriptors (SIFT, SURF, etc.). Thus, instead of depending on hand-crafted feature descriptors, we rely on the training of an image segmenter. The resulting map takes up much less storage space compared to a traditional descriptor based map. A particle filter based semantic localization solution is compared to one based on SIFT-features, and even with large seasonal variations over the year we perform on par with the larger and more descriptive SIFT-features, and are able to localize with an error below 1 m most of the time.