# Towards calibrated and scalable uncertainty representations for neural networks

arXiv: https://arxiv.org/abs/1911.00104 · HTML: https://mlsanity.com/paper/1911.00104/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Oct 2019.

## Built on by (most cited first, 10 of 10)

- [The Unreasonable Effectiveness of Deep Evidential Regression](https://mlsanity.com/paper/2205.10060/) (2022)
- [Multivariate Deep Evidential Regression](https://mlsanity.com/paper/2104.06135/) (2021)
- [Data-IQ: Characterizing subgroups with heterogeneous outcomes in tabular data](https://mlsanity.com/paper/2210.13043/) (2022)
- [Data-SUITE: Data-centric identification of in-distribution incongruous examples](https://mlsanity.com/paper/2202.08836/) (2022)
- [A Structured Review of Literature on Uncertainty in Machine Learning & Deep Learning](https://mlsanity.com/paper/2406.00332/) (2024)
- [Promises and Pitfalls of Threshold-based Auto-labeling](https://mlsanity.com/paper/2211.12620/) (2022)
- [Aleatoric Uncertainty Medical Image Segmentation Estimation via Flow Matching](https://mlsanity.com/paper/2507.22418/) (2025)
- [Maximizing Uncertainty for Federated learning via Bayesian Optimisation-based Model Poisoning](https://mlsanity.com/paper/2501.08002/) (2025)
- [SACDNet: Towards Early Type 2 Diabetes Prediction with Uncertainty for Electronic Health Records](https://mlsanity.com/paper/2301.04844/) (2023)
- [Modeling Disagreement in Automatic Data Labelling for Semi-Supervised Learning in Clinical Natural Language Processing](https://mlsanity.com/paper/2205.14761/) (2022)

## Abstract (arXiv)

For many applications it is critical to know the uncertainty of a neural network's predictions. While a variety of neural network parameter estimation methods have been proposed for uncertainty estimation, they have not been rigorously compared across uncertainty measures. We assess four of these parameter estimation methods to calibrate uncertainty estimation using four different uncertainty measures: entropy, mutual information, aleatoric uncertainty and epistemic uncertainty. We evaluate the calibration of these parameter estimation methods using expected calibration error. Additionally, we propose a novel method of neural network parameter estimation called RECAST, which combines cosine annealing with warm restarts with Stochastic Gradient Langevin Dynamics, capturing more diverse parameter distributions. When benchmarked against mutilated image data, we show that RECAST is well-calibrated and when combined with predictive entropy and epistemic uncertainty it offers the best calibrated measure of uncertainty when compared to recent methods.