# RC-AutoCalib: An End-to-End Radar-Camera Automatic Calibration Network

arXiv: https://arxiv.org/abs/2505.22427 · HTML: https://mlsanity.com/paper/2505.22427/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 51 references that are in the map.
- First posted on arXiv 28 May 2025.

## Builds on (most cited first, 26 of 26)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data](https://mlsanity.com/paper/2401.10891/) (2024)
- [ZoeDepth: Zero-shot Transfer by Combining Relative and Metric Depth](https://mlsanity.com/paper/2302.12288/) (2023)
- [LightGlue: Local Feature Matching at Light Speed](https://mlsanity.com/paper/2306.13643/) (2023)
- [BEVDepth: Acquisition of Reliable Depth for Multi-view 3D Object Detection](https://mlsanity.com/paper/2206.10092/) (2022)
- [Empirical Evaluation of Rectified Activations in Convolutional Network](https://mlsanity.com/paper/1505.00853/) (2015)
- [Selective Kernel Networks](https://mlsanity.com/paper/1903.06586/) (2019)
- [Long Short-Term Memory Based Recurrent Neural Network Architectures for Large Vocabulary Speech Recognition](https://mlsanity.com/paper/1402.1128/) (2014)
- [Sparse Fuse Dense: Towards High Quality 3D Detection with Depth Completion](https://mlsanity.com/paper/2203.09780/) (2022)
- [CRN: Camera Radar Net for Accurate, Robust, Efficient 3D Perception](https://mlsanity.com/paper/2304.00670/) (2023)
- [RegNet: Multimodal Sensor Registration Using Deep Neural Networks](https://mlsanity.com/paper/1707.03167/) (2017)
- [Radar-Camera Pixel Depth Association for Depth Completion](https://mlsanity.com/paper/2106.02778/) (2021)
- [Pixel-level Extrinsic Self Calibration of High Resolution LiDAR and Camera in Targetless Environments](https://mlsanity.com/paper/2103.01627/) (2021)
- [RadarNet: Exploiting Radar for Robust Perception of Dynamic Objects](https://mlsanity.com/paper/2007.14366/) (2020)
- [aiMotive Dataset: A Multimodal Dataset for Robust Autonomous Driving with Long-Range Perception](https://mlsanity.com/paper/2211.09445/) (2022)
- [DXQ-Net: Differentiable LiDAR-Camera Extrinsic Calibration Using Quality-aware Flow](https://mlsanity.com/paper/2203.09385/) (2022)
- [CalibDNN: Multimodal Sensor Calibration for Perception Using Deep Neural Networks](https://mlsanity.com/paper/2103.14793/) (2021)
- [Targetless Rotational Auto-Calibration of Radar and Camera for Intelligent Transportation Systems](https://mlsanity.com/paper/1904.08743/) (2019)
- [CFNet: LiDAR-Camera Registration Using Calibration Flow Network](https://mlsanity.com/paper/2104.11907/) (2021)
- [Perceiving Longer Sequences With Bi-Directional Cross-Attention Transformers](https://mlsanity.com/paper/2402.12138/) (2024)
- [A Continuous-Time Approach for 3D Radar-to-Camera Extrinsic Calibration](https://mlsanity.com/paper/2103.07505/) (2021)
- [Spatiotemporal Calibration of 3D Millimetre-Wavelength Radar-Camera Pairs](https://mlsanity.com/paper/2211.01871/) (2022)

## Built on by (most cited first, 4 of 4)

- [DeepScholar-Bench: A Live Benchmark and Automated Evaluation for Generative Research Synthesis](https://mlsanity.com/paper/2508.20033/) (2025)
- [Controllable Radar Simulation with Waveform Parameter Embedding](https://mlsanity.com/paper/2506.03134/) (2025)
- [calibfusion: Transformer-Based Differentiable Calibration for Radar-Camera Fusion Detection in Water-Surface Environments](https://mlsanity.com/paper/2603.06670/) (2026)
- [Mind the Hitch: Dynamic Calibration and Articulated Perception for Autonomous Trucks](https://mlsanity.com/paper/2603.23711/) (2026)

## Abstract (arXiv)

This paper presents a groundbreaking approach - the first online automatic geometric calibration method for radar and camera systems. Given the significant data sparsity and measurement uncertainty in radar height data, achieving automatic calibration during system operation has long been a challenge. To address the sparsity issue, we propose a Dual-Perspective representation that gathers features from both frontal and bird's-eye views. The frontal view contains rich but sensitive height information, whereas the bird's-eye view provides robust features against height uncertainty. We thereby propose a novel Selective Fusion Mechanism to identify and fuse reliable features from both perspectives, reducing the effect of height uncertainty. Moreover, for each view, we incorporate a Multi-Modal Cross-Attention Mechanism to explicitly find location correspondences through cross-modal matching. During the training phase, we also design a Noise-Resistant Matcher to provide better supervision and enhance the robustness of the matching mechanism against sparsity and height uncertainty. Our experimental results, tested on the nuScenes dataset, demonstrate that our method significantly outperforms previous radar-camera auto-calibration methods, as well as existing state-of-the-art LiDAR-camera calibration techniques, establishing a new benchmark for future research. The code is available at https://github.com/nycu-acm/RC-AutoCalib.