# CalibDNN: Multimodal Sensor Calibration for Perception Using Deep Neural Networks

arXiv: https://arxiv.org/abs/2103.14793 · HTML: https://mlsanity.com/paper/2103.14793/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 28 references that are in the map.
- First posted on arXiv 27 Mar 2021.

## Builds on (most cited first, 15 of 15)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [A Point Set Generation Network for 3D Object Reconstruction from a Single Image](https://mlsanity.com/paper/1612.00603/) (2016)
- [TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems](https://mlsanity.com/paper/1603.04467/) (2016)
- [Network In Network](https://mlsanity.com/paper/1312.4400/) (2013)
- [RELLIS-3D Dataset: Data, Benchmarks and Analysis](https://mlsanity.com/paper/2011.12954/) (2020)
- [ACNet: Attention Based Network to Exploit Complementary Features for RGBD Semantic Segmentation](https://mlsanity.com/paper/1905.10089/) (2019)
- [RegNet: Multimodal Sensor Registration Using Deep Neural Networks](https://mlsanity.com/paper/1707.03167/) (2017)
- [Automatic Extrinsic Calibration for Lidar-Stereo Vehicle Sensor Setups](https://mlsanity.com/paper/1705.04085/) (2017)
- [LiDAR and Camera Calibration using Motion Estimated by Sensor Fusion Odometry](https://mlsanity.com/paper/1804.05178/) (2018)
- [Spatiotemporal Camera-LiDAR Calibration: A Targetless and Structureless Approach](https://mlsanity.com/paper/2001.06175/) (2020)
- [SOIC: Semantic Online Initialization and Calibration for LiDAR and Camera](https://mlsanity.com/paper/2003.04260/) (2020)
- [LCCNet: LiDAR and Camera Self-Calibration using Cost Volume Network](https://mlsanity.com/paper/2012.13901/) (2020)

## Built on by (most cited first, 14 of 14)

- [Radar-Camera Fusion for Object Detection and Semantic Segmentation in Autonomous Driving: A Comprehensive Review](https://mlsanity.com/paper/2304.10410/) (2023)
- [Radar and Camera Fusion for Object Detection and Tracking: A Comprehensive Survey](https://mlsanity.com/paper/2410.19872/) (2024)
- [SemCal: Semantic LiDAR-Camera Calibration using Neural MutualInformation Estimator](https://mlsanity.com/paper/2109.10270/) (2021)
- [From Chaos to Calibration: A Geometric Mutual Information Approach to Target-Free Camera LiDAR Extrinsic Calibration](https://mlsanity.com/paper/2311.01905/) (2023)
- [Enhanced Low-resolution LiDAR-Camera Calibration Via Depth Interpolation and Supervised Contrastive Learning](https://mlsanity.com/paper/2211.03932/) (2022)
- [RC-AutoCalib: An End-to-End Radar-Camera Automatic Calibration Network](https://mlsanity.com/paper/2505.22427/) (2025)
- [Environment-Driven Online LiDAR-Camera Extrinsic Calibration](https://mlsanity.com/paper/2502.00801/) (2025)
- [Improving Extrinsics between RADAR and LIDAR using Learning](https://mlsanity.com/paper/2305.10594/) (2023)
- [What Really Matters for Learning-based LiDAR-Camera Calibration](https://mlsanity.com/paper/2501.16969/) (2025)
- [Calibrating LiDAR and Camera using Semantic Mutual information](https://mlsanity.com/paper/2104.12023/) (2021)
- [DST-Calib: A Dual-Path, Self-Supervised, Target-Free LiDAR-Camera Extrinsic Calibration Network](https://mlsanity.com/paper/2601.01188/) (2026)
- [Evaluation of Multimodal Semantic Segmentation using RGB-D Data](https://mlsanity.com/paper/2103.16758/) (2021)
- [Continuous Online Extrinsic Calibration of Fisheye Camera and LiDAR](https://mlsanity.com/paper/2306.13240/) (2023)
- [Single-Frame Point-Pixel Registration via Supervised Cross-Modal Feature Matching](https://mlsanity.com/paper/2506.22784/) (2025)

## Abstract (arXiv)

Current perception systems often carry multimodal imagers and sensors such as 2D cameras and 3D LiDAR sensors. To fuse and utilize the data for downstream perception tasks, robust and accurate calibration of the multimodal sensor data is essential. We propose a novel deep learning-driven technique (CalibDNN) for accurate calibration among multimodal sensor, specifically LiDAR-Camera pairs. The key innovation of the proposed work is that it does not require any specific calibration targets or hardware assistants, and the entire processing is fully automatic with a single model and single iteration. Results comparison among different methods and extensive experiments on different datasets demonstrates the state-of-the-art performance.