# Pixel-level Extrinsic Self Calibration of High Resolution LiDAR and Camera in Targetless Environments

arXiv: https://arxiv.org/abs/2103.01627 · HTML: https://mlsanity.com/paper/2103.01627/

## Key facts

- Cited by 49 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Mar 2021.

## Built on by (most cited first, 30 of 49)

- [R3LIVE: A Robust, Real-time, RGB-colored, LiDAR-Inertial-Visual tightly-coupled state Estimation and mapping package](https://mlsanity.com/paper/2109.07982/) (2021)
- [InfraDet3D: Multi-Modal 3D Object Detection based on Roadside Infrastructure Camera and LiDAR Sensors](https://mlsanity.com/paper/2305.00314/) (2023)
- [STD: Stable Triangle Descriptor for 3D place recognition](https://mlsanity.com/paper/2209.12435/) (2022)
- [MOISST: Multimodal Optimization of Implicit Scene for SpatioTemporal calibration](https://mlsanity.com/paper/2303.03056/) (2023)
- [MMAUD: A Comprehensive Multi-Modal Anti-UAV Dataset for Modern Miniature Drone Threats](https://mlsanity.com/paper/2402.03706/) (2024)
- [SOAC: Spatio-Temporal Overlap-Aware Multi-Sensor Calibration using Neural Radiance Fields](https://mlsanity.com/paper/2311.15803/) (2023)
- [LCE-Calib: Automatic LiDAR-Frame/Event Camera Extrinsic Calibration With A Globally Optimal Solution](https://mlsanity.com/paper/2303.09825/) (2023)
- [CoFiI2P: Coarse-to-Fine Correspondences for Image-to-Point Cloud Registration](https://mlsanity.com/paper/2309.14660/) (2023)
- [3DGS-Calib: 3D Gaussian Splatting for Multimodal SpatioTemporal Calibration](https://mlsanity.com/paper/2403.11577/) (2024)
- [Online,Target-Free LiDAR-Camera Extrinsic Calibration via Cross-Modal Mask Matching](https://mlsanity.com/paper/2404.18083/) (2024)
- [CalibFormer: A Transformer-based Automatic LiDAR-Camera Calibration Network](https://mlsanity.com/paper/2311.15241/) (2023)
- [INF: Implicit Neural Fusion for LiDAR and Camera](https://mlsanity.com/paper/2308.14414/) (2023)
- [UniCal: Unified Neural Sensor Calibration](https://mlsanity.com/paper/2409.18953/) (2024)
- [UniCal: a Single-Branch Transformer-Based Model for Camera-to-LiDAR Calibration and Validation](https://mlsanity.com/paper/2304.09715/) (2023)
- [OmniColor: A Global Camera Pose Optimization Approach of LiDAR-360Camera Fusion for Colorizing Point Clouds](https://mlsanity.com/paper/2404.04693/) (2024)
- [Robust Long-Range Perception Against Sensor Misalignment in Autonomous Vehicles](https://mlsanity.com/paper/2408.11196/) (2024)
- [Physics and semantic informed multi-sensor calibration via optimization theory and self-supervised learning](https://mlsanity.com/paper/2206.02856/) (2022)
- [Semantic Segmentation of Fruits on Multi-sensor Fused Data in Natural Orchards](https://mlsanity.com/paper/2208.02483/) (2022)
- [A9 Intersection Dataset: All You Need for Urban 3D Camera-LiDAR Roadside Perception](https://mlsanity.com/paper/2306.09266/) (2023)
- [RC-AutoCalib: An End-to-End Radar-Camera Automatic Calibration Network](https://mlsanity.com/paper/2505.22427/) (2025)
- [Lidar with Velocity: Correcting Moving Objects Point Cloud Distortion from Oscillating Scanning Lidars by Fusion with Camera](https://mlsanity.com/paper/2111.09497/) (2021)
- [CMR-Agent: Learning a Cross-Modal Agent for Iterative Image-to-Point Cloud Registration](https://mlsanity.com/paper/2408.02394/) (2024)
- [Environment-Driven Online LiDAR-Camera Extrinsic Calibration](https://mlsanity.com/paper/2502.00801/) (2025)
- [CalibRefine: Deep Learning-Based Online Automatic Targetless LiDAR-Camera Calibration with Iterative and Attention-Driven Post-Refinement](https://mlsanity.com/paper/2502.17648/) (2025)
- [TEScalib: Targetless Extrinsic Self-Calibration of LiDAR and Stereo Camera for Automated Driving Vehicles with Uncertainty Analysis](https://mlsanity.com/paper/2202.13847/) (2022)
- [SceneCalib: Automatic Targetless Calibration of Cameras and Lidars in Autonomous Driving](https://mlsanity.com/paper/2304.05530/) (2023)
- [PseudoCal: Towards Initialisation-Free Deep Learning-Based Camera-LiDAR Self-Calibration](https://mlsanity.com/paper/2309.09855/) (2023)
- [EdgeCalib: Multi-Frame Weighted Edge Features for Automatic Targetless LiDAR-Camera Calibration](https://mlsanity.com/paper/2310.16629/) (2023)
- [P2O-Calib: Camera-LiDAR Calibration Using Point-Pair Spatial Occlusion Relationship](https://mlsanity.com/paper/2311.02413/) (2023)
- [TrafficLoc: Localizing Traffic Surveillance Cameras in 3D Scenes](https://mlsanity.com/paper/2412.10308/) (2024)

## Abstract (arXiv)

In this letter, we present a novel method for automatic extrinsic calibration of high-resolution LiDARs and RGB cameras in targetless environments. Our approach does not require checkerboards but can achieve pixel-level accuracy by aligning natural edge features in the two sensors. On the theory level, we analyze the constraints imposed by edge features and the sensitivity of calibration accuracy with respect to edge distribution in the scene. On the implementation level, we carefully investigate the physical measuring principles of LiDARs and propose an efficient and accurate LiDAR edge extraction method based on point cloud voxel cutting and plane fitting. Due to the edges' richness in natural scenes, we have carried out experiments in many indoor and outdoor scenes. The results show that this method has high robustness, accuracy, and consistency. It can promote the research and application of the fusion between LiDAR and camera. We have open-sourced our code on GitHub to benefit the community.