# Targetless LiDAR-Camera Calibration with Neural Gaussian Splatting

arXiv: https://arxiv.org/abs/2504.04597 · HTML: https://mlsanity.com/paper/2504.04597/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 47 references that are in the map.
- First posted on arXiv 6 Apr 2025.

## Builds on (most cited first, 26 of 26)

- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://mlsanity.com/paper/2105.15203/) (2021)
- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [Scaffold-GS: Structured 3D Gaussians for View-Adaptive Rendering](https://mlsanity.com/paper/2312.00109/) (2023)
- [KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3D](https://mlsanity.com/paper/2109.13410/) (2021)
- [Gaussian Splatting SLAM](https://mlsanity.com/paper/2312.06741/) (2023)
- [NoPe-NeRF: Optimising Neural Radiance Field with No Pose Prior](https://mlsanity.com/paper/2212.07388/) (2022)
- [Self-Calibrating Neural Radiance Fields](https://mlsanity.com/paper/2108.13826/) (2021)
- [COLMAP-Free 3D Gaussian Splatting](https://mlsanity.com/paper/2312.07504/) (2023)
- [OmniRe: Omni Urban Scene Reconstruction](https://mlsanity.com/paper/2408.16760/) (2024)
- [KISS-ICP: In Defense of Point-to-Point ICP -- Simple, Accurate, and Robust Registration If Done the Right Way](https://mlsanity.com/paper/2209.15397/) (2022)
- [Scene Coordinate Reconstruction: Posing of Image Collections via Incremental Learning of a Relocalizer](https://mlsanity.com/paper/2404.14351/) (2024)
- [RegNet: Multimodal Sensor Registration Using Deep Neural Networks](https://mlsanity.com/paper/1707.03167/) (2017)
- [FAST-LIVO2: Fast, Direct LiDAR-Inertial-Visual Odometry](https://mlsanity.com/paper/2408.14035/) (2024)
- [NeRF-LOAM: Neural Implicit Representation for Large-Scale Incremental LiDAR Odometry and Mapping](https://mlsanity.com/paper/2303.10709/) (2023)
- [MOISST: Multimodal Optimization of Implicit Scene for SpatioTemporal calibration](https://mlsanity.com/paper/2303.03056/) (2023)
- [PointNeRF++: A multi-scale, point-based Neural Radiance Field](https://mlsanity.com/paper/2312.02362/) (2023)
- [3DGS-Calib: 3D Gaussian Splatting for Multimodal SpatioTemporal Calibration](https://mlsanity.com/paper/2403.11577/) (2024)
- [TCLC-GS: Tightly Coupled LiDAR-Camera Gaussian Splatting for Autonomous Driving](https://mlsanity.com/paper/2404.02410/) (2024)
- [INF: Implicit Neural Fusion for LiDAR and Camera](https://mlsanity.com/paper/2308.14414/) (2023)
- [UniCal: Unified Neural Sensor Calibration](https://mlsanity.com/paper/2409.18953/) (2024)
- [Look Gauss, No Pose: Novel View Synthesis using Gaussian Splatting without Accurate Pose Initialization](https://mlsanity.com/paper/2410.08743/) (2024)
- [Temporal and Spatial Online Integrated Calibration for Camera and LiDAR](https://mlsanity.com/paper/2207.10454/) (2022)
- [Robust LiDAR-Camera Calibration with 2D Gaussian Splatting](https://mlsanity.com/paper/2504.00525/) (2025)

## Built on by (most cited first, 2 of 2)

- [Refining Ground Truth Poses in Autonomous Driving Datasets via Neural Rendering](https://mlsanity.com/paper/2504.15776/) (2025)
- [Geometry-Preserving in 3D Gaussian Splatting for LiDAR-Camera Extrinsic Calibration](https://mlsanity.com/paper/2606.20103/) (2026)

## Abstract (arXiv)

Accurate LiDAR-camera calibration is crucial for multi-sensor systems. However, traditional methods often rely on physical targets, which are impractical for real-world deployment. Moreover, even carefully calibrated extrinsics can degrade over time due to sensor drift or external disturbances, necessitating periodic recalibration. To address these challenges, we present a Targetless LiDAR-Camera Calibration (TLC-Calib) that jointly optimizes sensor poses with a neural Gaussian-based scene representation. Reliable LiDAR points are frozen as anchor Gaussians to preserve global structure, while auxiliary Gaussians prevent local overfitting under noisy initialization. Our fully differentiable pipeline with photometric and geometric regularization achieves robust and generalizable calibration, consistently outperforming existing targetless methods on the KITTI-360, Waymo, and Fast-LIVO2 datasets. In addition, it yields more consistent Novel View Synthesis results, reflecting improved extrinsic alignment. The project page is available at: https://www.haebeom.com/tlc-calib-site/.