# Monocular Localization with Semantics Map for Autonomous Vehicles

arXiv: https://arxiv.org/abs/2406.03835 · HTML: https://mlsanity.com/paper/2406.03835/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 37 references that are in the map.
- First posted on arXiv 6 Jun 2024.

## Builds on (most cited first, 17 of 17)

- [KPConv: Flexible and Deformable Convolution for Point Clouds](https://mlsanity.com/paper/1904.08889/) (2019)
- [ORB-SLAM: a Versatile and Accurate Monocular SLAM System](https://mlsanity.com/paper/1502.00956/) (2015)
- [ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual-Inertial and Multi-Map SLAM](https://mlsanity.com/paper/2007.11898/) (2020)
- [ORB-SLAM2: an Open-Source SLAM System for Monocular, Stereo and RGB-D Cameras](https://mlsanity.com/paper/1610.06475/) (2016)
- [From Coarse to Fine: Robust Hierarchical Localization at Large Scale](https://mlsanity.com/paper/1812.03506/) (2018)
- [VINS-Mono: A Robust and Versatile Monocular Visual-Inertial State Estimator](https://mlsanity.com/paper/1708.03852/) (2017)
- [BiSeNet V2: Bilateral Network with Guided Aggregation for Real-time Semantic Segmentation](https://mlsanity.com/paper/2004.02147/) (2020)
- [Back to the Feature: Learning Robust Camera Localization from Pixels to Pose](https://mlsanity.com/paper/2103.09213/) (2021)
- [FAST-LIO2: Fast Direct LiDAR-inertial Odometry](https://mlsanity.com/paper/2107.06829/) (2021)
- [CubeSLAM: Monocular 3D Object SLAM](https://mlsanity.com/paper/1806.00557/) (2018)
- [FAST-LIO: A Fast, Robust LiDAR-inertial Odometry Package by Tightly-Coupled Iterated Kalman Filter](https://mlsanity.com/paper/2010.08196/) (2020)
- [DynaSLAM II: Tightly-Coupled Multi-Object Tracking and SLAM](https://mlsanity.com/paper/2010.07820/) (2020)
- [Tightly Coupled 3D Lidar Inertial Odometry and Mapping](https://mlsanity.com/paper/1904.06993/) (2019)
- [RoadMap: A Light-Weight Semantic Map for Visual Localization towards Autonomous Driving](https://mlsanity.com/paper/2106.02527/) (2021)
- [DXSLAM: A Robust and Efficient Visual SLAM System with Deep Features](https://mlsanity.com/paper/2008.05416/) (2020)
- [Long-term Visual Localization using Semantically Segmented Images](https://mlsanity.com/paper/1801.05269/) (2018)
- [Coarse-To-Fine Visual Localization Using Semantic Compact Map](https://mlsanity.com/paper/1910.04936/) (2019)

## Built on by (most cited first, 1 of 1)

- [Map-Free Visual Relocalization Enhanced by Instance Knowledge and Depth Knowledge](https://mlsanity.com/paper/2408.13085/) (2024)

## Abstract (arXiv)

Accurate and robust localization remains a significant challenge for autonomous vehicles. The cost of sensors and limitations in local computational efficiency make it difficult to scale to large commercial applications. Traditional vision-based approaches focus on texture features that are susceptible to changes in lighting, season, perspective, and appearance. Additionally, the large storage size of maps with descriptors and complex optimization processes hinder system performance. To balance efficiency and accuracy, we propose a novel lightweight visual semantic localization algorithm that employs stable semantic features instead of low-level texture features. First, semantic maps are constructed offline by detecting semantic objects, such as ground markers, lane lines, and poles, using cameras or LiDAR sensors. Then, online visual localization is performed through data association of semantic features and map objects. We evaluated our proposed localization framework in the publicly available KAIST Urban dataset and in scenarios recorded by ourselves. The experimental results demonstrate that our method is a reliable and practical localization solution in various autonomous driving localization tasks.