# LCCNet: LiDAR and Camera Self-Calibration using Cost Volume Network

arXiv: https://arxiv.org/abs/2012.13901 · HTML: https://mlsanity.com/paper/2012.13901/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Dec 2020.

## Built on by (most cited first, 4 of 4)

- [CalibDNN: Multimodal Sensor Calibration for Perception Using Deep Neural Networks](https://mlsanity.com/paper/2103.14793/) (2021)
- [SemCal: Semantic LiDAR-Camera Calibration using Neural MutualInformation Estimator](https://mlsanity.com/paper/2109.10270/) (2021)
- [Physics and semantic informed multi-sensor calibration via optimization theory and self-supervised learning](https://mlsanity.com/paper/2206.02856/) (2022)
- [Calibrating LiDAR and Camera using Semantic Mutual information](https://mlsanity.com/paper/2104.12023/) (2021)

## Abstract (arXiv)

In this paper, we propose a novel online self-calibration approach for Light Detection and Ranging (LiDAR) and camera sensors. Compared to the previous CNN-based methods that concatenate the feature maps of the RGB image and decalibrated depth image, we exploit the cost volume inspired by the PWC-Net for feature matching. Besides the smooth L1-Loss of the predicted extrinsic calibration parameters, an additional point cloud loss is applied. Instead of regress the extrinsic parameters between LiDAR and camera directly, we predict the decalibrated deviation from initial calibration to the ground truth. During inference, the calibration error decreases further with the usage of iterative refinement and the temporal filtering approach. The evaluation results on the KITTI dataset illustrate that our approach outperforms CNN-based state-of-the-art methods in terms of a mean absolute calibration error of 0.297cm in translation and 0.017deg in rotation with miscalibration magnitudes of up to 1.5m and 20deg.