# EgoVM: Achieving Precise Ego-Localization using Lightweight Vectorized Maps

arXiv: https://arxiv.org/abs/2307.08991 · HTML: https://mlsanity.com/paper/2307.08991/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 67 references that are in the map.
- First posted on arXiv 18 Jul 2023.

## Builds on (most cited first, 30 of 32)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [PointPillars: Fast Encoders for Object Detection from Point Clouds](https://mlsanity.com/paper/1812.05784/) (2018)
- [BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformers](https://mlsanity.com/paper/2203.17270/) (2022)
- [Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3D](https://mlsanity.com/paper/2008.05711/) (2020)
- [BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representation](https://mlsanity.com/paper/2205.13542/) (2022)
- [BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-View](https://mlsanity.com/paper/2112.11790/) (2021)
- [BEVDepth: Acquisition of Reliable Depth for Multi-view 3D Object Detection](https://mlsanity.com/paper/2206.10092/) (2022)
- [TransFusion: Robust LiDAR-Camera Fusion for 3D Object Detection with Transformers](https://mlsanity.com/paper/2203.11496/) (2022)
- [PointPainting: Sequential Fusion for 3D Object Detection](https://mlsanity.com/paper/1911.10150/) (2019)
- [PETRv2: A Unified Framework for 3D Perception from Multi-Camera Images](https://mlsanity.com/paper/2206.01256/) (2022)
- [Categorical Depth Distribution Network for Monocular 3D Object Detection](https://mlsanity.com/paper/2103.01100/) (2021)
- [MapTR: Structured Modeling and Learning for Online Vectorized HD Map Construction](https://mlsanity.com/paper/2208.14437/) (2022)
- [Cross-view Transformers for real-time Map-view Semantic Segmentation](https://mlsanity.com/paper/2205.02833/) (2022)
- [VectorMapNet: End-to-end Vectorized HD Map Learning](https://mlsanity.com/paper/2206.08920/) (2022)
- [FIERY: Future Instance Prediction in Bird's-Eye View from Surround Monocular Cameras](https://mlsanity.com/paper/2104.10490/) (2021)
- [CenterMask : Real-Time Anchor-Free Instance Segmentation](https://mlsanity.com/paper/1911.06667/) (2019)
- [Self-Driving Cars: A Survey](https://mlsanity.com/paper/1901.04407/) (2019)
- [BEVSegFormer: Bird's Eye View Semantic Segmentation From Arbitrary Camera Rigs](https://mlsanity.com/paper/2203.04050/) (2022)
- [EPro-PnP: Generalized End-to-End Probabilistic Perspective-n-Points for Monocular Object Pose Estimation](https://mlsanity.com/paper/2303.12787/) (2023)
- [RoadMap: A Light-Weight Semantic Map for Visual Localization towards Autonomous Driving](https://mlsanity.com/paper/2106.02527/) (2021)
- [Learning to Localize Using a LiDAR Intensity Map](https://mlsanity.com/paper/2012.10902/) (2020)
- [Robust and Precise Vehicle Localization based on Multi-sensor Fusion in Diverse City Scenes](https://mlsanity.com/paper/1711.05805/) (2017)
- [Exploiting Sparse Semantic HD Maps for Self-Driving Vehicle Localization](https://mlsanity.com/paper/1908.03274/) (2019)
- [BEV-Locator: An End-to-end Visual Semantic Localization Network Using Multi-View Images](https://mlsanity.com/paper/2211.14927/) (2022)
- [Learning to Localize Through Compressed Binary Maps](https://mlsanity.com/paper/2012.10942/) (2020)
- [Robust Monocular Localization in Sparse HD Maps Leveraging Multi-Task Uncertainty Estimation](https://mlsanity.com/paper/2110.10563/) (2021)
- [Long-Term Urban Vehicle Localization Using Pole Landmarks Extracted from 3-D Lidar Scans](https://mlsanity.com/paper/1910.10550/) (2019)

## Built on by (most cited first, 11 of 11)

- [MGMap: Mask-Guided Learning for Online Vectorized HD Map Construction](https://mlsanity.com/paper/2404.00876/) (2024)
- [MapLocNet: Coarse-to-Fine Feature Registration for Visual Re-Localization in Navigation Maps](https://mlsanity.com/paper/2407.08561/) (2024)
- [SegLocNet: Multimodal Localization Network for Autonomous Driving via Bird's-Eye-View Segmentation](https://mlsanity.com/paper/2502.20077/) (2025)
- [RTMap: Real-Time Recursive Mapping with Change Detection and Localization](https://mlsanity.com/paper/2507.00980/) (2025)
- [U-ViLAR: Uncertainty-Aware Visual Localization for Autonomous Driving via Differentiable Association and Registration](https://mlsanity.com/paper/2507.04503/) (2025)
- [Astra: Toward General-Purpose Mobile Robots via Hierarchical Multimodal Learning](https://mlsanity.com/paper/2506.06205/) (2025)
- [Uncertainty-Instructed Structure Injection for Generalizable HD Map Construction](https://mlsanity.com/paper/2503.23109/) (2025)
- [HOLO: Homography-Guided Pose Estimator Network for Fine-Grained Visual Localization on SD Maps](https://mlsanity.com/paper/2601.02730/) (2026)
- [VVLoc: Prior-free 3-DoF Vehicle Visual Localization](https://mlsanity.com/paper/2602.00810/) (2026)
- [Robust Localization for Autonomous Vehicles in Highway Scenes](https://mlsanity.com/paper/2604.22040/) (2026)
- [GSMap: 2D Gaussians for Online HD Mapping](https://mlsanity.com/paper/2605.09619/) (2026)

## Abstract (arXiv)

Accurate and reliable ego-localization is critical for autonomous driving. In this paper, we present EgoVM, an end-to-end localization network that achieves comparable localization accuracy to prior state-of-the-art methods, but uses lightweight vectorized maps instead of heavy point-based maps. To begin with, we extract BEV features from online multi-view images and LiDAR point cloud. Then, we employ a set of learnable semantic embeddings to encode the semantic types of map elements and supervise them with semantic segmentation, to make their feature representation consistent with BEV features. After that, we feed map queries, composed of learnable semantic embeddings and coordinates of map elements, into a transformer decoder to perform cross-modality matching with BEV features. Finally, we adopt a robust histogram-based pose solver to estimate the optimal pose by searching exhaustively over candidate poses. We comprehensively validate the effectiveness of our method using both the nuScenes dataset and a newly collected dataset. The experimental results show that our method achieves centimeter-level localization accuracy, and outperforms existing methods using vectorized maps by a large margin. Furthermore, our model has been extensively tested in a large fleet of autonomous vehicles under various challenging urban scenes.