# Pointing the Way: Refining Radar-Lidar Localization Using Learned ICP Weights

arXiv: https://arxiv.org/abs/2309.08731 · HTML: https://mlsanity.com/paper/2309.08731/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 50 references that are in the map.
- First posted on arXiv 15 Sep 2023.

## Builds on (most cited first, 24 of 24)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables](https://mlsanity.com/paper/1611.00712/) (2016)
- [Boreas: A Multi-Season Autonomous Driving Dataset](https://mlsanity.com/paper/2203.10168/) (2022)
- [A Survey on Global LiDAR Localization: Challenges, Advances and Open Problems](https://mlsanity.com/paper/2302.07433/) (2023)
- [PyPose: A Library for Robot Learning with Physics-based Optimization](https://mlsanity.com/paper/2209.15428/) (2022)
- [A New Wave in Robotics: Survey on Recent mmWave Radar Applications in Robotics](https://mlsanity.com/paper/2305.01135/) (2023)
- [Under the Radar: Learning to Predict Robust Keypoints for Odometry Estimation and Metric Localisation in Radar](https://mlsanity.com/paper/2001.10789/) (2020)
- [Localization Requirements for Autonomous Vehicles](https://mlsanity.com/paper/1906.01061/) (2019)
- [Masking by Moving: Learning Distraction-Free Radar Odometry from Pose Information](https://mlsanity.com/paper/1909.03752/) (2019)
- [Deep Probabilistic Feature-metric Tracking](https://mlsanity.com/paper/2008.13504/) (2020)
- [There and Back Again: Learning to Simulate Radar Data for Real-World Applications](https://mlsanity.com/paper/2011.14389/) (2020)
- [Radar Odometry Combining Probabilistic Estimation and Unsupervised Feature Learning](https://mlsanity.com/paper/2105.14152/) (2021)
- [Analysis of Robust Functions for Registration Algorithms](https://mlsanity.com/paper/1810.01474/) (2018)
- [Are We Ready for Radar to Replace Lidar in All-Weather Mapping and Localization?](https://mlsanity.com/paper/2203.10174/) (2022)
- [RaLL: End-to-end Radar Localization on Lidar Map Using Differentiable Measurement Model](https://mlsanity.com/paper/2009.07061/) (2020)
- [Radar-to-Lidar: Heterogeneous Place Recognition via Joint Learning](https://mlsanity.com/paper/2102.04960/) (2021)
- [Radar Odometry for Autonomous Ground Vehicles: A Survey of Methods and Datasets](https://mlsanity.com/paper/2307.07861/) (2023)
- [Safe Autonomous Driving in Adverse Weather: Sensor Evaluation and Performance Monitoring](https://mlsanity.com/paper/2305.01336/) (2023)
- [Radar-on-Lidar: metric radar localization on prior lidar maps](https://mlsanity.com/paper/2005.04644/) (2020)
- [On the Importance of Quantifying Visibility for Autonomous Vehicles under Extreme Precipitation](https://mlsanity.com/paper/2209.03226/) (2022)
- [BFAR-Bounded False Alarm Rate detector for improved radar odometry estimation](https://mlsanity.com/paper/2109.09669/) (2021)

## Built on by (most cited first, 1 of 1)

- [Prepared for the Worst: A Learning-Based Adversarial Attack for Resilience Analysis of the ICP Algorithm](https://mlsanity.com/paper/2403.05666/) (2024)

## Abstract (arXiv)

This paper presents a novel deep-learning-based approach to improve localizing radar measurements against lidar maps. This radar-lidar localization leverages the benefits of both sensors; radar is resilient against adverse weather, while lidar produces high-quality maps in clear conditions. However, owing in part to the unique artefacts present in radar measurements, radar-lidar localization has struggled to achieve comparable performance to lidar-lidar systems, preventing it from being viable for autonomous driving. This work builds on ICP-based radar-lidar localization by including a learned preprocessing step that weights radar points based on high-level scan information. To train the weight-generating network, we present a novel, stand-alone, open-source differentiable ICP library. The learned weights facilitate ICP by filtering out harmful radar points related to artefacts, noise, and even vehicles on the road. Combining an analytical approach with a learned weight reduces overall localization errors and improves convergence in radar-lidar ICP results run on real-world autonomous driving data. Our code base is publicly available to facilitate reproducibility and extensions.