# SOIC: Semantic Online Initialization and Calibration for LiDAR and Camera

arXiv: https://arxiv.org/abs/2003.04260 · HTML: https://mlsanity.com/paper/2003.04260/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Mar 2020.

## Built on by (most cited first, 15 of 15)

- [Deep Learning for Camera Calibration and Beyond: A Survey](https://mlsanity.com/paper/2303.10559/) (2023)
- [CRLF: Automatic Calibration and Refinement based on Line Feature for LiDAR and Camera in Road Scenes](https://mlsanity.com/paper/2103.04558/) (2021)
- [CalibDNN: Multimodal Sensor Calibration for Perception Using Deep Neural Networks](https://mlsanity.com/paper/2103.14793/) (2021)
- [Calib-Anything: Zero-training LiDAR-Camera Extrinsic Calibration Method Using Segment Anything](https://mlsanity.com/paper/2306.02656/) (2023)
- [SemCal: Semantic LiDAR-Camera Calibration using Neural MutualInformation Estimator](https://mlsanity.com/paper/2109.10270/) (2021)
- [Physics and semantic informed multi-sensor calibration via optimization theory and self-supervised learning](https://mlsanity.com/paper/2206.02856/) (2022)
- [TEScalib: Targetless Extrinsic Self-Calibration of LiDAR and Stereo Camera for Automated Driving Vehicles with Uncertainty Analysis](https://mlsanity.com/paper/2202.13847/) (2022)
- [Improving Extrinsics between RADAR and LIDAR using Learning](https://mlsanity.com/paper/2305.10594/) (2023)
- [EdgeCalib: Multi-Frame Weighted Edge Features for Automatic Targetless LiDAR-Camera Calibration](https://mlsanity.com/paper/2310.16629/) (2023)
- [Calibrating LiDAR and Camera using Semantic Mutual information](https://mlsanity.com/paper/2104.12023/) (2021)
- [RobustCalib: Robust Lidar-Camera Extrinsic Calibration with Consistency Learning](https://mlsanity.com/paper/2312.01085/) (2023)
- [Cooperative Visual-LiDAR Extrinsic Calibration Technology for Intersection Vehicle-Infrastructure: A review](https://mlsanity.com/paper/2405.10132/) (2024)
- [Online LiDAR-Camera Extrinsic Parameters Self-checking](https://mlsanity.com/paper/2210.10537/) (2022)
- [EEPNet-V2: Patch-to-Pixel Solution for Efficient Cross-Modal Registration between LiDAR Point Cloud and Camera Image](https://mlsanity.com/paper/2503.15285/) (2025)
- [Targetless 6DoF Calibration of LiDAR and 2D Scanning Radar Based on Cylindrical Occupancy](https://mlsanity.com/paper/2503.17002/) (2025)

## Abstract (arXiv)

This paper presents a novel semantic-based online extrinsic calibration approach, SOIC (so, I see), for Light Detection and Ranging (LiDAR) and camera sensors. Previous online calibration methods usually need prior knowledge of rough initial values for optimization. The proposed approach removes this limitation by converting the initialization problem to a Perspective-n-Point (PnP) problem with the introduction of semantic centroids (SCs). The closed-form solution of this PnP problem has been well researched and can be found with existing PnP methods. Since the semantic centroid of the point cloud usually does not accurately match with that of the corresponding image, the accuracy of parameters are not improved even after a nonlinear refinement process. Thus, a cost function based on the constraint of the correspondence between semantic elements from both point cloud and image data is formulated. Subsequently, optimal extrinsic parameters are estimated by minimizing the cost function. We evaluate the proposed method either with GT or predicted semantics on KITTI dataset. Experimental results and comparisons with the baseline method verify the feasibility of the initialization strategy and the accuracy of the calibration approach. In addition, we release the source code at https://github.com/--/SOIC.