# Coarse-to-fine Semantic Localization with HD Map for Autonomous Driving in Structural Scenes

arXiv: https://arxiv.org/abs/2107.02557 · HTML: https://mlsanity.com/paper/2107.02557/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 31 references that are in the map.
- First posted on arXiv 6 Jul 2021.

## Builds on (most cited first, 12 of 12)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [SuperPoint: Self-Supervised Interest Point Detection and Description](https://mlsanity.com/paper/1712.07629/) (2017)
- [ORB-SLAM2: an Open-Source SLAM System for Monocular, Stereo and RGB-D Cameras](https://mlsanity.com/paper/1610.06475/) (2016)
- [From Coarse to Fine: Robust Hierarchical Localization at Large Scale](https://mlsanity.com/paper/1812.03506/) (2018)
- [Direct Sparse Odometry](https://mlsanity.com/paper/1607.02565/) (2016)
- [Back to the Feature: Learning Robust Camera Localization from Pixels to Pose](https://mlsanity.com/paper/2103.09213/) (2021)
- [Exploiting Sparse Semantic HD Maps for Self-Driving Vehicle Localization](https://mlsanity.com/paper/1908.03274/) (2019)
- [Monocular Camera Localization in Prior LiDAR Maps with 2D-3D Line Correspondences](https://mlsanity.com/paper/2004.00740/) (2020)
- [AVP-SLAM: Semantic Visual Mapping and Localization for Autonomous Vehicles in the Parking Lot](https://mlsanity.com/paper/2007.01813/) (2020)
- [Pop-up SLAM: Semantic Monocular Plane SLAM for Low-texture Environments](https://mlsanity.com/paper/1703.07334/) (2017)
- [Monocular Vehicle Self-localization method based on Compact Semantic Map](https://mlsanity.com/paper/1805.06155/) (2018)

## Built on by (most cited first, 5 of 5)

- [OrienterNet: Visual Localization in 2D Public Maps with Neural Matching](https://mlsanity.com/paper/2304.02009/) (2023)
- [InstaGraM: Instance-level Graph Modeling for Vectorized HD Map Learning](https://mlsanity.com/paper/2301.04470/) (2023)
- [Convolutional Cross-View Pose Estimation](https://mlsanity.com/paper/2303.05915/) (2023)
- [EgoVM: Achieving Precise Ego-Localization using Lightweight Vectorized Maps](https://mlsanity.com/paper/2307.08991/) (2023)
- [Robust Vehicle Localization and Tracking in Rain using Street Maps](https://mlsanity.com/paper/2409.01038/) (2024)

## Abstract (arXiv)

Robust and accurate localization is an essential component for robotic navigation and autonomous driving. The use of cameras for localization with high definition map (HD Map) provides an affordable localization sensor set. Existing methods suffer from pose estimation failure due to error prone data association or initialization with accurate initial pose requirement. In this paper, we propose a cost-effective vehicle localization system with HD map for autonomous driving that uses cameras as primary sensors. To this end, we formulate vision-based localization as a data association problem that maps visual semantics to landmarks in HD map. Specifically, system initialization is finished in a coarse to fine manner by combining coarse GPS (Global Positioning System) measurement and fine pose searching. In tracking stage, vehicle pose is refined by implicitly aligning the semantic segmentation result between image and landmarks in HD maps with photometric consistency. Finally, vehicle pose is computed by pose graph optimization in a sliding window fashion. We evaluate our method on two datasets and demonstrate that the proposed approach yields promising localization results in different driving scenarios. Additionally, our approach is suitable for both monocular camera and multi-cameras that provides flexibility and improves robustness for the localization system.