# Estimating and Evaluating Regression Predictive Uncertainty in Deep Object Detectors

arXiv: https://arxiv.org/abs/2101.05036 · HTML: https://mlsanity.com/paper/2101.05036/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 46 references that are in the map.
- First posted on arXiv 13 Jan 2021.

## Builds on (most cited first, 29 of 29)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift](https://mlsanity.com/paper/1906.02530/) (2019)
- [The Open Images Dataset V4: Unified image classification, object detection, and visual relationship detection at scale](https://mlsanity.com/paper/1811.00982/) (2018)
- [Accurate Uncertainties for Deep Learning Using Calibrated Regression](https://mlsanity.com/paper/1807.00263/) (2018)
- [Verified Uncertainty Calibration](https://mlsanity.com/paper/1909.10155/) (2019)
- [Generative Moment Matching Networks](https://mlsanity.com/paper/1502.02761/) (2015)
- [MMD GAN: Towards Deeper Understanding of Moment Matching Network](https://mlsanity.com/paper/1705.08584/) (2017)
- [Pitfalls of In-Domain Uncertainty Estimation and Ensembling in Deep Learning](https://mlsanity.com/paper/2002.06470/) (2020)
- [Bounding Box Regression with Uncertainty for Accurate Object Detection](https://mlsanity.com/paper/1809.08545/) (2018)
- [LaserNet: An Efficient Probabilistic 3D Object Detector for Autonomous Driving](https://mlsanity.com/paper/1903.08701/) (2019)
- [The Cramer Distance as a Solution to Biased Wasserstein Gradients](https://mlsanity.com/paper/1705.10743/) (2017)
- [Gaussian YOLOv3: An Accurate and Fast Object Detector Using Localization Uncertainty for Autonomous Driving](https://mlsanity.com/paper/1904.04620/) (2019)
- [Towards Safe Autonomous Driving: Capture Uncertainty in the Deep Neural Network For Lidar 3D Vehicle Detection](https://mlsanity.com/paper/1804.05132/) (2018)
- [BayesOD: A Bayesian Approach for Uncertainty Estimation in Deep Object Detectors](https://mlsanity.com/paper/1903.03838/) (2019)
- [Reliable training and estimation of variance networks](https://mlsanity.com/paper/1906.03260/) (2019)
- [Probabilistic Object Detection: Definition and Evaluation](https://mlsanity.com/paper/1811.10800/) (2018)
- [A Spectral Energy Distance for Parallel Speech Synthesis](https://mlsanity.com/paper/2008.01160/) (2020)
- [Uncertainty Estimation in One-Stage Object Detection](https://mlsanity.com/paper/1905.10296/) (2019)
- [Can We Trust You? On Calibration of a Probabilistic Object Detector for Autonomous Driving](https://mlsanity.com/paper/1909.12358/) (2019)
- [Leveraging Heteroscedastic Aleatoric Uncertainties for Robust Real-Time LiDAR 3D Object Detection](https://mlsanity.com/paper/1809.05590/) (2018)
- [Localization Uncertainty Estimation for Anchor-Free Object Detection](https://mlsanity.com/paper/2006.15607/) (2020)
- [DISCO Nets: DISsimilarity COefficient Networks](https://mlsanity.com/paper/1606.02556/) (2016)
- [A Mask-RCNN Baseline for Probabilistic Object Detection](https://mlsanity.com/paper/1908.03621/) (2019)

## Built on by (most cited first, 30 of 37)

- [VOS: Learning What You Don't Know by Virtual Outlier Synthesis](https://mlsanity.com/paper/2202.01197/) (2022)
- [On the Pitfalls of Heteroscedastic Uncertainty Estimation with Probabilistic Neural Networks](https://mlsanity.com/paper/2203.09168/) (2022)
- [Unknown-Aware Object Detection: Learning What You Don't Know from Videos in the Wild](https://mlsanity.com/paper/2203.03800/) (2022)
- [Uncertainty Quantification of Collaborative Detection for Self-Driving](https://mlsanity.com/paper/2209.08162/) (2022)
- [Probabilistic Forecasting with Generative Networks via Scoring Rule Minimization](https://mlsanity.com/paper/2112.08217/) (2021)
- [SAFE: Sensitivity-Aware Features for Out-of-Distribution Object Detection](https://mlsanity.com/paper/2208.13930/) (2022)
- [Collaborative Multi-Object Tracking with Conformal Uncertainty Propagation](https://mlsanity.com/paper/2303.14346/) (2023)
- [Uncertainty for Identifying Open-Set Errors in Visual Object Detection](https://mlsanity.com/paper/2104.01328/) (2021)
- [Towards Building Self-Aware Object Detectors via Reliable Uncertainty Quantification and Calibration](https://mlsanity.com/paper/2307.00934/) (2023)
- [MoCaE: Mixture of Calibrated Experts Significantly Improves Object Detection](https://mlsanity.com/paper/2309.14976/) (2023)
- [Normalizing Flow based Feature Synthesis for Outlier-Aware Object Detection](https://mlsanity.com/paper/2302.07106/) (2023)
- [Likelihood-Free Inference with Generative Neural Networks via Scoring Rule Minimization](https://mlsanity.com/paper/2205.15784/) (2022)
- [On Calibration of Object Detectors: Pitfalls, Evaluation and Baselines](https://mlsanity.com/paper/2405.20459/) (2024)
- [UncTrack: Reliable Visual Object Tracking with Uncertainty-Aware Prototype Memory Network](https://mlsanity.com/paper/2503.12888/) (2025)
- [Multi-view 3D Object Reconstruction and Uncertainty Modelling with Neural Shape Prior](https://mlsanity.com/paper/2306.11739/) (2023)
- [Beyond Classification: Definition and Density-based Estimation of Calibration in Object Detection](https://mlsanity.com/paper/2312.06645/) (2023)
- [Towards PAC Multi-Object Detection and Tracking](https://mlsanity.com/paper/2204.07482/) (2022)
- [Uncertainty-aware 3D Object-Level Mapping with Deep Shape Priors](https://mlsanity.com/paper/2309.09118/) (2023)
- [DrIFT: Autonomous Drone Dataset with Integrated Real and Synthetic Data, Flexible Views, and Transformed Domains](https://mlsanity.com/paper/2412.04789/) (2024)
- [LiDAR-MIMO: Efficient Uncertainty Estimation for LiDAR-based 3D Object Detection](https://mlsanity.com/paper/2206.00214/) (2022)
- [Estimating Regression Predictive Distributions with Sample Networks](https://mlsanity.com/paper/2211.13724/) (2022)
- [UncertaintyTrack: Exploiting Detection and Localization Uncertainty in Multi-Object Tracking](https://mlsanity.com/paper/2402.12303/) (2024)
- [Object Detection as Probabilistic Set Prediction](https://mlsanity.com/paper/2203.07980/) (2022)
- [Sample-based Uncertainty Quantification with a Single Deterministic Neural Network](https://mlsanity.com/paper/2209.08418/) (2022)
- [Query-based Hard-Image Retrieval for Object Detection at Test Time](https://mlsanity.com/paper/2209.11559/) (2022)
- [ProPanDL: A Modular Architecture for Uncertainty-Aware Panoptic Segmentation](https://mlsanity.com/paper/2304.08645/) (2023)
- [FEVER-OOD: Free Energy Vulnerability Elimination for Robust Out-of-Distribution Detection](https://mlsanity.com/paper/2412.01596/) (2024)
- [Efficient Epistemic Uncertainty Estimation in Regression Ensemble Models Using Pairwise-Distance Estimators](https://mlsanity.com/paper/2308.13498/) (2023)
- [Out-of-distribution Object Detection through Bayesian Uncertainty Estimation](https://mlsanity.com/paper/2310.19119/) (2023)
- [Self-Aware Object Detection via Degradation Manifolds](https://mlsanity.com/paper/2602.18394/) (2026)

## Abstract (arXiv)

Predictive uncertainty estimation is an essential next step for the reliable deployment of deep object detectors in safety-critical tasks. In this work, we focus on estimating predictive distributions for bounding box regression output with variance networks. We show that in the context of object detection, training variance networks with negative log likelihood (NLL) can lead to high entropy predictive distributions regardless of the correctness of the output mean. We propose to use the energy score as a non-local proper scoring rule and find that when used for training, the energy score leads to better calibrated and lower entropy predictive distributions than NLL. We also address the widespread use of non-proper scoring metrics for evaluating predictive distributions from deep object detectors by proposing an alternate evaluation approach founded on proper scoring rules. Using the proposed evaluation tools, we show that although variance networks can be used to produce high quality predictive distributions, ad-hoc approaches used by seminal object detectors for choosing regression targets during training do not provide wide enough data support for reliable variance learning. We hope that our work helps shift evaluation in probabilistic object detection to better align with predictive uncertainty evaluation in other machine learning domains. Code for all models, evaluation, and datasets is available at: https://github.com/asharakeh/probdet.git.