# RaLL: End-to-end Radar Localization on Lidar Map Using Differentiable Measurement Model

arXiv: https://arxiv.org/abs/2009.07061 · HTML: https://mlsanity.com/paper/2009.07061/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Sep 2020.

## Built on by (most cited first, 8 of 8)

- [Visual Cross-View Metric Localization with Dense Uncertainty Estimates](https://mlsanity.com/paper/2208.08519/) (2022)
- [General Place Recognition Survey: Towards the Real-world Autonomy Age](https://mlsanity.com/paper/2209.04497/) (2022)
- [Radar-to-Lidar: Heterogeneous Place Recognition via Joint Learning](https://mlsanity.com/paper/2102.04960/) (2021)
- [RaLF: Flow-based Global and Metric Radar Localization in LiDAR Maps](https://mlsanity.com/paper/2309.09875/) (2023)
- [Learning Interpretable BEV Based VIO without Deep Neural Networks](https://mlsanity.com/paper/2109.12292/) (2021)
- [Pointing the Way: Refining Radar-Lidar Localization Using Learned ICP Weights](https://mlsanity.com/paper/2309.08731/) (2023)
- [Improved Radar Localization on Lidar Maps Using Shared Embedding](https://mlsanity.com/paper/2106.10000/) (2021)
- [CoFi: Coarse-to-Fine ICP for LiDAR Localization in an Efficient Long-lasting Point Cloud Map](https://mlsanity.com/paper/2110.10194/) (2021)

## Abstract (arXiv)

Compared to the onboard camera and laser scanner, radar sensor provides lighting and weather invariant sensing, which is naturally suitable for long-term localization under adverse conditions. However, radar data is sparse and noisy, resulting in challenges for radar mapping. On the other hand, the most popular available map currently is built by lidar. In this paper, we propose an end-to-end deep learning framework for Radar Localization on Lidar Map (RaLL) to bridge the gap, which not only achieves the robust radar localization but also exploits the mature lidar mapping technique, thus reducing the cost of radar mapping. We first embed both sensor modals into a common feature space by a neural network. Then multiple offsets are added to the map modal for exhaustive similarity evaluation against the current radar modal, yielding the regression of the current pose. Finally, we apply this differentiable measurement model to a Kalman Filter (KF) to learn the whole sequential localization process in an end-to-end manner. textitThe whole learning system is differentiable with the network based measurement model at the front-end and KF at the back-end. To validate the feasibility and effectiveness, we employ multi-session multi-scene datasets collected from the real world, and the results demonstrate that our proposed system achieves superior performance over 90km driving, even in generalization scenarios where the model training is in UK, while testing in South Korea. We also release the source code publicly.