# S-BEV: Semantic Birds-Eye View Representation for Weather and Lighting Invariant 3-DoF Localization

arXiv: https://arxiv.org/abs/2101.09569 · HTML: https://mlsanity.com/paper/2101.09569/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 24 references that are in the map.
- First posted on arXiv 23 Jan 2021.

## Builds on (most cited first, 14 of 14)

- [SuperPoint: Self-Supervised Interest Point Detection and Description](https://mlsanity.com/paper/1712.07629/) (2017)
- [Depth Map Prediction from a Single Image using a Multi-Scale Deep Network](https://mlsanity.com/paper/1406.2283/) (2014)
- [Semantic Understanding of Scenes through the ADE20K Dataset](https://mlsanity.com/paper/1608.05442/) (2016)
- [From Coarse to Fine: Robust Hierarchical Localization at Large Scale](https://mlsanity.com/paper/1812.03506/) (2018)
- [Virtual KITTI 2](https://mlsanity.com/paper/2001.10773/) (2020)
- [Virtual Worlds as Proxy for Multi-Object Tracking Analysis](https://mlsanity.com/paper/1605.06457/) (2016)
- [DeMoN: Depth and Motion Network for Learning Monocular Stereo](https://mlsanity.com/paper/1612.02401/) (2016)
- [UnDeepVO: Monocular Visual Odometry through Unsupervised Deep Learning](https://mlsanity.com/paper/1709.06841/) (2017)
- [Semantic Visual Localization](https://mlsanity.com/paper/1712.05773/) (2017)
- [Ford Multi-AV Seasonal Dataset](https://mlsanity.com/paper/2003.07969/) (2020)
- [BEV-Seg: Bird's Eye View Semantic Segmentation Using Geometry and Semantic Point Cloud](https://mlsanity.com/paper/2006.11436/) (2020)
- [Delta Descriptors: Change-Based Place Representation for Robust Visual Localization](https://mlsanity.com/paper/2006.05700/) (2020)
- [Long-term Visual Localization using Semantically Segmented Images](https://mlsanity.com/paper/1801.05269/) (2018)
- [Deep-Geometric 6 DoF Localization from a Single Image in Topo-metric Maps](https://mlsanity.com/paper/2002.01210/) (2020)

## Built on by (most cited first, 3 of 3)

- [CVLNet: Cross-View Semantic Correspondence Learning for Video-based Camera Localization](https://mlsanity.com/paper/2208.03660/) (2022)
- [Improving Worst Case Visual Localization Coverage via Place-specific Sub-selection in Multi-camera Systems](https://mlsanity.com/paper/2206.13883/) (2022)
- [Stereo Visual Odometry with Deep Learning-Based Point and Line Feature Matching using an Attention Graph Neural Network](https://mlsanity.com/paper/2308.01125/) (2023)

## Abstract (arXiv)

We describe a light-weight, weather and lighting invariant, Semantic Bird's Eye View (S-BEV) signature for vision-based vehicle re-localization. A topological map of S-BEV signatures is created during the first traversal of the route, which are used for coarse localization in subsequent route traversal. A fine-grained localizer is then trained to output the global 3-DoF pose of the vehicle using its S-BEV and its coarse localization. We conduct experiments on vKITTI2 virtual dataset and show the potential of the S-BEV to be robust to weather and lighting. We also demonstrate results with 2 vehicles on a 22 km long highway route in the Ford AV dataset.