# Multiclass Confidence and Localization Calibration for Object Detection

arXiv: https://arxiv.org/abs/2306.08271 · HTML: https://mlsanity.com/paper/2306.08271/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 53 references that are in the map.
- First posted on arXiv 14 Jun 2023.

## Builds on (most cited first, 30 of 37)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [FCOS: Fully Convolutional One-Stage Object Detection](https://mlsanity.com/paper/1904.01355/) (2019)
- [BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learning](https://mlsanity.com/paper/1805.04687/) (2018)
- [Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift](https://mlsanity.com/paper/1906.02530/) (2019)
- [When Does Label Smoothing Help?](https://mlsanity.com/paper/1906.02629/) (2019)
- [Regularizing Neural Networks by Penalizing Confident Output Distributions](https://mlsanity.com/paper/1701.06548/) (2017)
- [Semantic Foggy Scene Understanding with Synthetic Data](https://mlsanity.com/paper/1708.07819/) (2017)
- [A Survey of Deep Learning Techniques for Autonomous Driving](https://mlsanity.com/paper/1910.07738/) (2019)
- [Calibrating Deep Neural Networks using Focal Loss](https://mlsanity.com/paper/2002.09437/) (2020)
- [Probabilistic Backpropagation for Scalable Learning of Bayesian Neural Networks](https://mlsanity.com/paper/1502.05336/) (2015)
- [Cross-Domain Weakly-Supervised Object Detection through Progressive Domain Adaptation](https://mlsanity.com/paper/1803.11365/) (2018)
- [Driving in the Matrix: Can Virtual Worlds Replace Human-Generated Annotations for Real World Tasks?](https://mlsanity.com/paper/1610.01983/) (2016)
- [Training independent subnetworks for robust prediction](https://mlsanity.com/paper/2010.06610/) (2020)
- [Hyperparameter Ensembles for Robustness and Uncertainty Quantification](https://mlsanity.com/paper/2006.13570/) (2020)
- [Improving model calibration with accuracy versus uncertainty optimization](https://mlsanity.com/paper/2012.07923/) (2020)
- [Every Pixel Matters: Center-aware Feature Alignment for Domain Adaptive Object Detector](https://mlsanity.com/paper/2008.08574/) (2020)
- [Multivariate Confidence Calibration for Object Detection](https://mlsanity.com/paper/2004.13546/) (2020)
- [The Devil is in the Margin: Margin-based Label Smoothing for Network Calibration](https://mlsanity.com/paper/2111.15430/) (2021)

## Built on by (most cited first, 16 of 16)

- [MoCaE: Mixture of Calibrated Experts Significantly Improves Object Detection](https://mlsanity.com/paper/2309.14976/) (2023)
- [On Calibration of Object Detectors: Pitfalls, Evaluation and Baselines](https://mlsanity.com/paper/2405.20459/) (2024)
- [On the Calibration of Human Pose Estimation](https://mlsanity.com/paper/2311.17105/) (2023)
- [Beyond Classification: Definition and Density-based Estimation of Calibration in Object Detection](https://mlsanity.com/paper/2312.06645/) (2023)
- [Fractal Calibration for long-tailed object detection](https://mlsanity.com/paper/2410.11774/) (2024)
- [Uncertainty Quantification in Detection Transformers: Object-Level Calibration and Image-Level Reliability](https://mlsanity.com/paper/2412.01782/) (2024)
- [A Theoretical and Practical Framework for Evaluating Uncertainty Calibration in Object Detection](https://mlsanity.com/paper/2309.00464/) (2023)
- [Detecting Out-of-Distribution Objects through Class-Conditioned Inpainting](https://mlsanity.com/paper/2402.03292/) (2024)
- [A Decision-driven Methodology for Designing Uncertainty-aware AI Self-Assessment](https://mlsanity.com/paper/2408.01301/) (2024)
- [Exploring Scale Shift in Crowd Localization under the Context of Domain Generalization](https://mlsanity.com/paper/2510.19330/) (2025)
- [Query2Uncertainty: Robust Uncertainty Quantification and Calibration for 3D Object Detection under Distribution Shift](https://mlsanity.com/paper/2605.05328/) (2026)
- [Automated Model Evaluation for Object Detection via Prediction Consistency and Reliability](https://mlsanity.com/paper/2508.12082/) (2025)
- [Calibrating the Full Predictive Class Distribution of 3D Object Detectors for Autonomous Driving](https://mlsanity.com/paper/2510.01829/) (2025)
- [Calibrating Probabilistic Object Detectors with Annotator Disagreement](https://mlsanity.com/paper/2605.24722/) (2026)
- [Fair Classification with Efficient and Post-hoc Controllable Fairness-Accuracy Trade-off](https://mlsanity.com/paper/2606.28097/) (2026)
- [Rethinking Detection Calibration: A Coordinate and Direction Perspective](https://mlsanity.com/paper/2607.29040/) (2026)

## Abstract (arXiv)

Albeit achieving high predictive accuracy across many challenging computer vision problems, recent studies suggest that deep neural networks (DNNs) tend to make overconfident predictions, rendering them poorly calibrated. Most of the existing attempts for improving DNN calibration are limited to classification tasks and restricted to calibrating in-domain predictions. Surprisingly, very little to no attempts have been made in studying the calibration of object detection methods, which occupy a pivotal space in vision-based security-sensitive, and safety-critical applications. In this paper, we propose a new train-time technique for calibrating modern object detection methods. It is capable of jointly calibrating multiclass confidence and box localization by leveraging their predictive uncertainties. We perform extensive experiments on several in-domain and out-of-domain detection benchmarks. Results demonstrate that our proposed train-time calibration method consistently outperforms several baselines in reducing calibration error for both in-domain and out-of-domain predictions. Our code and models are available at https://github.com/bimsarapathiraja/MCCL.