# SST-Calib: Simultaneous Spatial-Temporal Parameter Calibration between LIDAR and Camera

arXiv: https://arxiv.org/abs/2207.03704 · HTML: https://mlsanity.com/paper/2207.03704/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 31 references that are in the map.
- First posted on arXiv 8 Jul 2022.

## Builds on (most cited first, 10 of 10)

- [Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3D](https://mlsanity.com/paper/2008.05711/) (2020)
- [Argoverse: 3D Tracking and Forecasting with Rich Maps](https://mlsanity.com/paper/1911.02620/) (2019)
- [SqueezeSegV3: Spatially-Adaptive Convolution for Efficient Point-Cloud Segmentation](https://mlsanity.com/paper/2004.01803/) (2020)
- [RegNet: Multimodal Sensor Registration Using Deep Neural Networks](https://mlsanity.com/paper/1707.03167/) (2017)
- [SDCNet: Video Prediction Using Spatially-Displaced Convolution](https://mlsanity.com/paper/1811.00684/) (2018)
- [RoarNet: A Robust 3D Object Detection based on RegiOn Approximation Refinement](https://mlsanity.com/paper/1811.03818/) (2018)
- [UrbanLoco: A Full Sensor Suite Dataset for Mapping and Localization in Urban Scenes](https://mlsanity.com/paper/1912.09513/) (2019)
- [Spatiotemporal Camera-LiDAR Calibration: A Targetless and Structureless Approach](https://mlsanity.com/paper/2001.06175/) (2020)
- [Automatic Construction of Lane-level HD Maps for Urban Scenes](https://mlsanity.com/paper/2107.10972/) (2021)
- [Advanced Mapping Robot and High-Resolution Dataset](https://mlsanity.com/paper/2007.12497/) (2020)

## Built on by (most cited first, 6 of 6)

- [Deep Learning for Camera Calibration and Beyond: A Survey](https://mlsanity.com/paper/2303.10559/) (2023)
- [SOAC: Spatio-Temporal Overlap-Aware Multi-Sensor Calibration using Neural Radiance Fields](https://mlsanity.com/paper/2311.15803/) (2023)
- [End-to-End Lidar-Camera Self-Calibration for Autonomous Vehicles](https://mlsanity.com/paper/2304.12412/) (2023)
- [Cooperative Visual-LiDAR Extrinsic Calibration Technology for Intersection Vehicle-Infrastructure: A review](https://mlsanity.com/paper/2405.10132/) (2024)
- [A re-calibration method for object detection with multi-modal alignment bias in autonomous driving](https://mlsanity.com/paper/2405.16848/) (2024)
- [DAP-Pose: Deep Temporal Alignment and Physics-aware Cross-modal Sensor Fusion for Robust Pose Estimation](https://mlsanity.com/paper/2607.23755/) (2026)

## Abstract (arXiv)

With information from multiple input modalities, sensor fusion-based algorithms usually out-perform their single-modality counterparts in robotics. Camera and LIDAR, with complementary semantic and depth information, are the typical choices for detection tasks in complicated driving environments. For most camera-LIDAR fusion algorithms, however, the calibration of the sensor suite will greatly impact the performance. More specifically, the detection algorithm usually requires an accurate geometric relationship among multiple sensors as the input, and it is often assumed that the contents from these sensors are captured at the same time. Preparing such sensor suites involves carefully designed calibration rigs and accurate synchronization mechanisms, and the preparation process is usually done offline. In this work, a segmentation-based framework is proposed to jointly estimate the geometrical and temporal parameters in the calibration of a camera-LIDAR suite. A semantic segmentation mask is first applied to both sensor modalities, and the calibration parameters are optimized through pixel-wise bidirectional loss. We specifically incorporated the velocity information from optical flow for temporal parameters. Since supervision is only performed at the segmentation level, no calibration label is needed within the framework. The proposed algorithm is tested on the KITTI dataset, and the result shows an accurate real-time calibration of both geometric and temporal parameters.