# Extrinsic Camera Calibration with Semantic Segmentation

arXiv: https://arxiv.org/abs/2208.03949 · HTML: https://mlsanity.com/paper/2208.03949/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 27 references that are in the map.
- First posted on arXiv 8 Aug 2022.

## Builds on (most cited first, 13 of 13)

- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [Accelerating 3D Deep Learning with PyTorch3D](https://mlsanity.com/paper/2007.08501/) (2020)
- [Segmentation Transformer: Object-Contextual Representations for Semantic Segmentation](https://mlsanity.com/paper/1909.11065/) (2019)
- [Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR Segmentation](https://mlsanity.com/paper/2011.10033/) (2020)
- [2D3D-MatchNet: Learning to Match Keypoints Across 2D Image and 3D Point Cloud](https://mlsanity.com/paper/1904.09742/) (2019)
- [DeepI2P: Image-to-Point Cloud Registration via Deep Classification](https://mlsanity.com/paper/2104.03501/) (2021)
- [Traffic Control Gesture Recognition for Autonomous Vehicles](https://mlsanity.com/paper/2007.16072/) (2020)
- [Monocular Camera Localization in Prior LiDAR Maps with 2D-3D Line Correspondences](https://mlsanity.com/paper/2004.00740/) (2020)
- [DeepLocalization: Landmark-based Self-Localization with Deep Neural Networks](https://mlsanity.com/paper/1904.09007/) (2019)
- [Certifiably Optimal Monocular Hand-Eye Calibration](https://mlsanity.com/paper/2005.08298/) (2020)
- [Online Extrinsic Calibration based on Per-Sensor Ego-Motion Using Dual Quaternions](https://mlsanity.com/paper/2101.11440/) (2021)
- [LACI: Low-effort Automatic Calibration of Infrastructure Sensors](https://mlsanity.com/paper/1911.01711/) (2019)

## Built on by (most cited first, 6 of 6)

- [TUMTraf Event: Calibration and Fusion Resulting in a Dataset for Roadside Event-Based and RGB Cameras](https://mlsanity.com/paper/2401.08474/) (2024)
- [Calib-Anything: Zero-training LiDAR-Camera Extrinsic Calibration Method Using Segment Anything](https://mlsanity.com/paper/2306.02656/) (2023)
- [Automated Static Camera Calibration with Intelligent Vehicles](https://mlsanity.com/paper/2304.10814/) (2023)
- [RobustCalib: Robust Lidar-Camera Extrinsic Calibration with Consistency Learning](https://mlsanity.com/paper/2312.01085/) (2023)
- [A re-calibration method for object detection with multi-modal alignment bias in autonomous driving](https://mlsanity.com/paper/2405.16848/) (2024)
- [Automated Automotive Radar Calibration With Intelligent Vehicles](https://mlsanity.com/paper/2306.13323/) (2023)

## Abstract (arXiv)

Monocular camera sensors are vital to intelligent vehicle operation and automated driving assistance and are also heavily employed in traffic control infrastructure. Calibrating the monocular camera, though, is time-consuming and often requires significant manual intervention. In this work, we present an extrinsic camera calibration approach that automatizes the parameter estimation by utilizing semantic segmentation information from images and point clouds. Our approach relies on a coarse initial measurement of the camera pose and builds on lidar sensors mounted on a vehicle with high-precision localization to capture a point cloud of the camera environment. Afterward, a mapping between the camera and world coordinate spaces is obtained by performing a lidar-to-camera registration of the semantically segmented sensor data. We evaluate our method on simulated and real-world data to demonstrate low error measurements in the calibration results. Our approach is suitable for infrastructure sensors as well as vehicle sensors, while it does not require motion of the camera platform.