# Online,Target-Free LiDAR-Camera Extrinsic Calibration via Cross-Modal Mask Matching

arXiv: https://arxiv.org/abs/2404.18083 · HTML: https://mlsanity.com/paper/2404.18083/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 54 references that are in the map.
- First posted on arXiv 28 Apr 2024.

## Builds on (most cited first, 28 of 28)

- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [SuperGlue: Learning Feature Matching with Graph Neural Networks](https://mlsanity.com/paper/1911.11763/) (2019)
- [TransFusion: Robust LiDAR-Camera Fusion for 3D Object Detection with Transformers](https://mlsanity.com/paper/2203.11496/) (2022)
- [Faster Segment Anything: Towards Lightweight SAM for Mobile Applications](https://mlsanity.com/paper/2306.14289/) (2023)
- [DeepFusion: Lidar-Camera Deep Fusion for Multi-Modal 3D Object Detection](https://mlsanity.com/paper/2203.08195/) (2022)
- [Deep Learning for Image and Point Cloud Fusion in Autonomous Driving: A Review](https://mlsanity.com/paper/2004.05224/) (2020)
- [RegNet: Multimodal Sensor Registration Using Deep Neural Networks](https://mlsanity.com/paper/1707.03167/) (2017)
- [SNE-RoadSeg: Incorporating Surface Normal Information into Semantic Segmentation for Accurate Freespace Detection](https://mlsanity.com/paper/2008.11351/) (2020)
- [Pixel-level Extrinsic Self Calibration of High Resolution LiDAR and Camera in Targetless Environments](https://mlsanity.com/paper/2103.01627/) (2021)
- [General, Single-shot, Target-less, and Automatic LiDAR-Camera Extrinsic Calibration Toolbox](https://mlsanity.com/paper/2302.05094/) (2023)
- [Pothole Detection Based on Disparity Transformation and Road Surface Modeling](https://mlsanity.com/paper/1908.00894/) (2019)
- [EDTER: Edge Detection with Transformer](https://mlsanity.com/paper/2203.08566/) (2022)
- [Road surface 3d reconstruction based on dense subpixel disparity map estimation](https://mlsanity.com/paper/1807.01874/) (2018)
- [RoadFormer: Duplex Transformer for RGB-Normal Semantic Road Scene Parsing](https://mlsanity.com/paper/2309.10356/) (2023)
- [Automatic Extrinsic Calibration Method for LiDAR and Camera Sensor Setups](https://mlsanity.com/paper/2101.04431/) (2021)
- [Playing to Vision Foundation Model's Strengths in Stereo Matching](https://mlsanity.com/paper/2404.06261/) (2024)
- [CRLF: Automatic Calibration and Refinement based on Line Feature for LiDAR and Camera in Road Scenes](https://mlsanity.com/paper/2103.04558/) (2021)
- [Computer Vision for Road Imaging and Pothole Detection: A State-of-the-Art Review of Systems and Algorithms](https://mlsanity.com/paper/2204.13590/) (2022)
- [LiDAR-Camera Calibration using 3D-3D Point correspondences](https://mlsanity.com/paper/1705.09785/) (2017)
- [Joint Camera Intrinsic and LiDAR-Camera Extrinsic Calibration](https://mlsanity.com/paper/2202.13708/) (2022)
- [S3M-Net: Joint Learning of Semantic Segmentation and Stereo Matching for Autonomous Driving](https://mlsanity.com/paper/2401.11414/) (2024)
- [Optimising the selection of samples for robust lidar camera calibration](https://mlsanity.com/paper/2103.12287/) (2021)
- [ACSC: Automatic Calibration for Non-repetitive Scanning Solid-State LiDAR and Camera Systems](https://mlsanity.com/paper/2011.08516/) (2020)
- [Auto-calibration Method Using Stop Signs for Urban Autonomous Driving Applications](https://mlsanity.com/paper/2010.07441/) (2020)
- [Dive Deeper into Rectifying Homography for Stereo Camera Online Self-Calibration](https://mlsanity.com/paper/2309.10314/) (2023)
- [Temporal and Spatial Online Integrated Calibration for Camera and LiDAR](https://mlsanity.com/paper/2207.10454/) (2022)
- [Multi-FEAT: Multi-Feature Edge Alignment for Targetless Camera-LiDAR Calibration](https://mlsanity.com/paper/2207.07228/) (2022)

## Built on by (most cited first, 12 of 12)

- [DCPI-Depth: Explicitly Infusing Dense Correspondence Prior to Unsupervised Monocular Depth Estimation](https://mlsanity.com/paper/2405.16960/) (2024)
- [SCIPaD: Incorporating Spatial Clues into Unsupervised Pose-Depth Joint Learning](https://mlsanity.com/paper/2407.05283/) (2024)
- [Environment-Driven Online LiDAR-Camera Extrinsic Calibration](https://mlsanity.com/paper/2502.00801/) (2025)
- [Iterative Camera-LiDAR Extrinsic Optimization via Surrogate Diffusion](https://mlsanity.com/paper/2411.10936/) (2024)
- [CLAIM: Camera-LiDAR Alignment with Intensity and Monodepth](https://mlsanity.com/paper/2512.14001/) (2025)
- [DST-Calib: A Dual-Path, Self-Supervised, Target-Free LiDAR-Camera Extrinsic Calibration Network](https://mlsanity.com/paper/2601.01188/) (2026)
- [Fully Exploiting Vision Foundation Model's Profound Prior Knowledge for Generalizable RGB-Depth Driving Scene Parsing](https://mlsanity.com/paper/2502.06219/) (2025)
- [Iterative Camera-LiDAR Extrinsic Optimization via Surrogate Diffusion](https://mlsanity.com/paper/2506.14706/) (2025)
- [Single-Frame Point-Pixel Registration via Supervised Cross-Modal Feature Matching](https://mlsanity.com/paper/2506.22784/) (2025)
- [RAVES-Calib: Robust, Accurate and Versatile Extrinsic Self Calibration Using Optimal Geometric Features](https://mlsanity.com/paper/2512.08170/) (2025)
- [Native-Domain Cross-Attention for Camera-LiDAR Extrinsic Calibration Under Large Initial Perturbations](https://mlsanity.com/paper/2603.29414/) (2026)
- [CalibBEV: LiDAR-Camera Calibration via BEV Alignment](https://mlsanity.com/paper/2608.02309/) (2026)

## Abstract (arXiv)

LiDAR-camera extrinsic calibration (LCEC) is crucial for data fusion in intelligent vehicles. Offline, target-based approaches have long been the preferred choice in this field. However, they often demonstrate poor adaptability to real-world environments. This is largely because extrinsic parameters may change significantly due to moderate shocks or during extended operations in environments with vibrations. In contrast, online, target-free approaches provide greater adaptability yet typically lack robustness, primarily due to the challenges in cross-modal feature matching. Therefore, in this article, we unleash the full potential of large vision models (LVMs), which are emerging as a significant trend in the fields of computer vision and robotics, especially for embodied artificial intelligence, to achieve robust and accurate online, target-free LCEC across a variety of challenging scenarios. Our main contributions are threefold: we introduce a novel framework known as MIAS-LCEC, provide an open-source versatile calibration toolbox with an interactive visualization interface, and publish three real-world datasets captured from various indoor and outdoor environments. The cornerstone of our framework and toolbox is the cross-modal mask matching (C3M) algorithm, developed based on a state-of-the-art (SoTA) LVM and capable of generating sufficient and reliable matches. Extensive experiments conducted on these real-world datasets demonstrate the robustness of our approach and its superior performance compared to SoTA methods, particularly for the solid-state LiDARs with super-wide fields of view.