# Calibrating Perception Uncertainty for Autonomous Driving

arXiv: https://arxiv.org/abs/2211.04340 · HTML: https://mlsanity.com/paper/2211.04340/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in End-to-end system architecture, one of 9 directions in End-to-end autonomous driving.
- Builds on 18 of its 39 references that are in the map.
- First posted on arXiv 8 Nov 2022.

## Placed in

- End-to-end autonomous driving / [End-to-end system architecture](https://mlsanity.com/driving/end-to-end-system-architecture/)

## Builds on (most cited first, 18 of 18)

- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [Concrete Problems in AI Safety](https://mlsanity.com/paper/1606.06565/) (2016)
- [Accurate Uncertainties for Deep Learning Using Calibrated Regression](https://mlsanity.com/paper/1807.00263/) (2018)
- [MultiPath: Multiple Probabilistic Anchor Trajectory Hypotheses for Behavior Prediction](https://mlsanity.com/paper/1910.05449/) (2019)
- [A Survey of Motion Planning and Control Techniques for Self-driving Urban Vehicles](https://mlsanity.com/paper/1604.07446/) (2016)
- [FIERY: Future Instance Prediction in Bird's-Eye View from Surround Monocular Cameras](https://mlsanity.com/paper/2104.10490/) (2021)
- [Evaluating Scalable Bayesian Deep Learning Methods for Robust Computer Vision](https://mlsanity.com/paper/1906.01620/) (2019)
- [Evaluating Bayesian Deep Learning Methods for Semantic Segmentation](https://mlsanity.com/paper/1811.12709/) (2018)
- [Rules of the Road: Predicting Driving Behavior with a Convolutional Model of Semantic Interactions](https://mlsanity.com/paper/1906.08945/) (2019)
- [Gaussian YOLOv3: An Accurate and Fast Object Detector Using Localization Uncertainty for Autonomous Driving](https://mlsanity.com/paper/1904.04620/) (2019)
- [Multi-Modal Trajectory Prediction of Surrounding Vehicles with Maneuver based LSTMs](https://mlsanity.com/paper/1805.05499/) (2018)
- [Safe Reinforcement Learning with Model Uncertainty Estimates](https://mlsanity.com/paper/1810.08700/) (2018)
- [Can We Trust You? On Calibration of a Probabilistic Object Detector for Autonomous Driving](https://mlsanity.com/paper/1909.12358/) (2019)
- [Labels Are Not Perfect: Inferring Spatial Uncertainty in Object Detection](https://mlsanity.com/paper/2012.12195/) (2020)
- [Moment-Sum-Of-Squares Approach For Fast Risk Estimation In Uncertain Environments](https://mlsanity.com/paper/1810.01577/) (2018)
- [On the Usefulness of the Fit-on-the-Test View on Evaluating Calibration of Classifiers](https://mlsanity.com/paper/2203.08958/) (2022)
- [Fast nonlinear risk assessment for autonomous vehicles using learned conditional probabilistic models of agent futures](https://mlsanity.com/paper/2109.09975/) (2021)

## Built on by (most cited first, 3 of 3)

- [ImageNet-D: Benchmarking Neural Network Robustness on Diffusion Synthetic Object](https://mlsanity.com/paper/2403.18775/) (2024)
- [MapPrior: Bird's-Eye View Map Layout Estimation with Generative Models](https://mlsanity.com/paper/2308.12963/) (2023)
- [Predictive Uncertainty Quantification for Bird's Eye View Segmentation: A Benchmark and Novel Loss Function](https://mlsanity.com/paper/2405.20986/) (2024)

## Abstract (arXiv)

Autonomous driving systems must be capable of making quick decisions based on the perceived environment and specific driving conditions. Perception models in these systems perform well in detecting objects under favourable conditions but their performance deteriorates in poor visibility or with partly occluded objects. To reduce risks from undetected objects, autonomous vehicles must incorporate all relevant uncertainties into their decision-making processes. Grid-based perception outputs, such as occupancy grids, and object-based outputs, like lists of detected objects, must be accompanied by well-calibrated uncertainty estimates. These uncertainties are essential for ensuring the model's reliability and safety. In this paper, we identify limitations in the current state-of-the-art and propose a more comprehensive set of uncertainty estimates that should be reported. In addition to commonly estimated forms of uncertainty about the presence, location, shape, and trajectory of detected objects, we propose to quantify the uncertainty about undetected objects within a region. Access to this set of uncertainties enables planners to perform region occupancy queries, which provide the probability that a certain region, such as the area around a chosen trajectory, is free of obstacles. We propose a novel approach for generating these probabilistic outputs from bird's-eye-view (BEV) probabilistic semantic segmentation. Our experiments demonstrate that the initial probabilistic outputs from segmentation are not calibrated, and we present methods to achieve well-calibrated uncertainty estimates. Finally, we conduct an experiment on a downstream task, underscoring the importance of calibrated uncertainties for planning and highlighting the advantages of including additional uncertainty types.