# CLRNet: Targetless Extrinsic Calibration for Camera, Lidar and 4D Radar Using Deep Learning

arXiv: https://arxiv.org/abs/2603.15767 · HTML: https://mlsanity.com/paper/2603.15767/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 39 references that are in the map.
- First posted on arXiv 16 Mar 2026.

## Builds on (most cited first, 16 of 16)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Depth Anything V2](https://mlsanity.com/paper/2406.09414/) (2024)
- [PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume](https://mlsanity.com/paper/1709.02371/) (2017)
- [RegNet: Multimodal Sensor Registration Using Deep Neural Networks](https://mlsanity.com/paper/1707.03167/) (2017)
- [Pixel-level Extrinsic Self Calibration of High Resolution LiDAR and Camera in Targetless Environments](https://mlsanity.com/paper/2103.01627/) (2021)
- [Dual Radar: A Multi-modal Dataset with Dual 4D Radar for Autonomous Driving](https://mlsanity.com/paper/2310.07602/) (2023)
- [DXQ-Net: Differentiable LiDAR-Camera Extrinsic Calibration Using Quality-aware Flow](https://mlsanity.com/paper/2203.09385/) (2022)
- [CMRNext: Camera to LiDAR Matching in the Wild for Localization and Extrinsic Calibration](https://mlsanity.com/paper/2402.00129/) (2024)
- [Targetless Rotational Auto-Calibration of Radar and Camera for Intelligent Transportation Systems](https://mlsanity.com/paper/1904.08743/) (2019)
- [Online Targetless Radar-Camera Extrinsic Calibration Based on the Common Features of Radar and Camera](https://mlsanity.com/paper/2309.00787/) (2023)
- [iKalibr: Unified Targetless Spatiotemporal Calibration for Resilient Integrated Inertial Systems](https://mlsanity.com/paper/2407.11420/) (2024)
- [Spatiotemporal Calibration of 3D Millimetre-Wavelength Radar-Camera Pairs](https://mlsanity.com/paper/2211.01871/) (2022)
- [Physics and semantic informed multi-sensor calibration via optimization theory and self-supervised learning](https://mlsanity.com/paper/2206.02856/) (2022)
- [SensorX2car: Sensors-to-car calibration for autonomous driving in road scenarios](https://mlsanity.com/paper/2301.07279/) (2023)
- [BEVCALIB: LiDAR-Camera Calibration via Geometry-Guided Bird's-Eye View Representations](https://mlsanity.com/paper/2506.02587/) (2025)
- [What Really Matters for Learning-based LiDAR-Camera Calibration](https://mlsanity.com/paper/2501.16969/) (2025)

## Abstract (arXiv)

In this paper, we address extrinsic calibration for camera, lidar, and 4D radar sensors. Accurate extrinsic calibration of radar remains a challenge due to the sparsity of its data. We propose CLRNet, a novel, multi-modal end-to-end deep learning (DL) calibration network capable of addressing joint camera-lidar-radar calibration, or pairwise calibration between any two of these sensors. We incorporate equirectangular projection, camera-based depth image prediction, additional radar channels, and leverage lidar with a shared feature space and loop closure loss. In extensive experiments using the View-of-Delft and Dual-Radar datasets, we demonstrate superior calibration accuracy compared to existing state-of-the-art methods, reducing both median translational and rotational calibration errors by at least 50%. Finally, we examine the domain transfer capabilities of the proposed network and baselines, when evaluating across datasets. The code will be made publicly available upon acceptance at: https://github.com/tudelft-iv.