# Calib-Anything: Zero-training LiDAR-Camera Extrinsic Calibration Method Using Segment Anything

arXiv: https://arxiv.org/abs/2306.02656 · HTML: https://mlsanity.com/paper/2306.02656/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 36 references that are in the map.
- First posted on arXiv 5 Jun 2023.

## Builds on (most cited first, 11 of 11)

- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [DeepFusion: Lidar-Camera Deep Fusion for Multi-Modal 3D Object Detection](https://mlsanity.com/paper/2203.08195/) (2022)
- [Self-supervised Sparse-to-Dense: Self-supervised Depth Completion from LiDAR and Monocular Camera](https://mlsanity.com/paper/1807.00275/) (2018)
- [RegNet: Multimodal Sensor Registration Using Deep Neural Networks](https://mlsanity.com/paper/1707.03167/) (2017)
- [Automatic Extrinsic Calibration Method for LiDAR and Camera Sensor Setups](https://mlsanity.com/paper/2101.04431/) (2021)
- [CRLF: Automatic Calibration and Refinement based on Line Feature for LiDAR and Camera in Road Scenes](https://mlsanity.com/paper/2103.04558/) (2021)
- [LiDAR and Camera Calibration using Motion Estimated by Sensor Fusion Odometry](https://mlsanity.com/paper/1804.05178/) (2018)
- [Automatic extrinsic calibration between a camera and a 3D Lidar using 3D point and plane correspondences](https://mlsanity.com/paper/1904.12433/) (2019)
- [Spatiotemporal Camera-LiDAR Calibration: A Targetless and Structureless Approach](https://mlsanity.com/paper/2001.06175/) (2020)
- [SOIC: Semantic Online Initialization and Calibration for LiDAR and Camera](https://mlsanity.com/paper/2003.04260/) (2020)
- [Extrinsic Camera Calibration with Semantic Segmentation](https://mlsanity.com/paper/2208.03949/) (2022)

## Built on by (most cited first, 11 of 11)

- [Forging Vision Foundation Models for Autonomous Driving: Challenges, Methodologies, and Opportunities](https://mlsanity.com/paper/2401.08045/) (2024)
- [UrbanSAM: Learning Invariance-Inspired Adapters for Segment Anything Models in Urban Construction](https://mlsanity.com/paper/2502.15199/) (2025)
- [Industrial Segment Anything -- a Case Study in Aircraft Manufacturing, Intralogistics, Maintenance, Repair, and Overhaul](https://mlsanity.com/paper/2307.12674/) (2023)
- [Timealign: A multi-modal object detection method for time misalignment fusing in autonomous driving](https://mlsanity.com/paper/2412.10033/) (2024)
- [MamV2XCalib: V2X-based Target-less Infrastructure Camera Calibration with State Space Model](https://mlsanity.com/paper/2507.23595/) (2025)
- [A re-calibration method for object detection with multi-modal alignment bias in autonomous driving](https://mlsanity.com/paper/2405.16848/) (2024)
- [CLAIM: Camera-LiDAR Alignment with Intensity and Monodepth](https://mlsanity.com/paper/2512.14001/) (2025)
- [Framework and Multi-modal Dataset for Roadwork Zone Detection and Geo-localization](https://mlsanity.com/paper/2607.04330/) (2026)
- [Validation & Exploration of Multimodal Deep-Learning Camera-Lidar Calibration models](https://mlsanity.com/paper/2409.13402/) (2024)
- [IRisPath: Enhancing Costmap for Off-Road Navigation with Robust IR-RGB Fusion for Improved Day and Night Traversability](https://mlsanity.com/paper/2412.03173/) (2024)
- [Native-Domain Cross-Attention for Camera-LiDAR Extrinsic Calibration Under Large Initial Perturbations](https://mlsanity.com/paper/2603.29414/) (2026)

## Abstract (arXiv)

The research on extrinsic calibration between Light Detection and Ranging(LiDAR) and camera are being promoted to a more accurate, automatic and generic manner. Since deep learning has been employed in calibration, the restrictions on the scene are greatly reduced. However, data driven method has the drawback of low transfer-ability. It cannot adapt to dataset variations unless additional training is taken. With the advent of foundation model, this problem can be significantly mitigated. By using the Segment Anything Model(SAM), we propose a novel LiDAR-camera calibration method, which requires zero extra training and adapts to common scenes. With an initial guess, we opimize the extrinsic parameter by maximizing the consistency of points that are projected inside each image mask. The consistency includes three properties of the point cloud: the intensity, normal vector and categories derived from some segmentation methods. The experiments on different dataset have demonstrated the generality and comparable accuracy of our method. The code is available at https://github.com/OpenCalib/CalibAnything.