# Uncertainty Estimates of Predictions via a General Bias-Variance Decomposition

arXiv: https://arxiv.org/abs/2210.12256 · HTML: https://mlsanity.com/paper/2210.12256/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 60 references that are in the map.
- First posted on arXiv 21 Oct 2022.

## Builds on (most cited first, 16 of 16)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift](https://mlsanity.com/paper/1906.02530/) (2019)
- [Aleatoric uncertainty estimation with test-time augmentation for medical image segmentation with convolutional neural networks](https://mlsanity.com/paper/1807.07356/) (2018)
- [Pitfalls of In-Domain Uncertainty Estimation and Ensembling in Deep Learning](https://mlsanity.com/paper/2002.06470/) (2020)
- [Rethinking Bias-Variance Trade-off for Generalization of Neural Networks](https://mlsanity.com/paper/2002.11328/) (2020)
- [Understanding Double Descent Requires a Fine-Grained Bias-Variance Decomposition](https://mlsanity.com/paper/2011.03321/) (2020)
- [Towards Trustworthy Predictions from Deep Neural Networks with Fast Adversarial Calibration](https://mlsanity.com/paper/2012.10923/) (2020)
- [Understanding the bias-variance tradeoff of Bregman divergences](https://mlsanity.com/paper/2202.04167/) (2022)
- [Ensembling over Classifiers: a Bias-Variance Perspective](https://mlsanity.com/paper/2206.10566/) (2022)

## Built on by (most cited first, 20 of 20)

- [Benchmarking Uncertainty Disentanglement: Specialized Uncertainties for Specialized Tasks](https://mlsanity.com/paper/2402.19460/) (2024)
- [From Risk to Uncertainty: Generating Predictive Uncertainty Measures via Bayesian Estimation](https://mlsanity.com/paper/2402.10727/) (2024)
- [Rethinking Uncertainty Estimation in LLMs: A Principled Single-Sequence Measure](https://mlsanity.com/paper/2412.15176/) (2024)
- [On Information-Theoretic Measures of Predictive Uncertainty](https://mlsanity.com/paper/2410.10786/) (2024)
- [Are We Done with Object-Centric Learning?](https://mlsanity.com/paper/2504.07092/) (2025)
- [Foundations of Top-k Decoding For Language Models](https://mlsanity.com/paper/2505.19371/) (2025)
- [Fine-Grained Uncertainty Decomposition in Large Language Models: A Spectral Approach](https://mlsanity.com/paper/2509.22272/) (2025)
- [Position: Epistemic uncertainty estimation methods are fundamentally incomplete](https://mlsanity.com/paper/2505.23506/) (2025)
- [A Bias-Variance-Covariance Decomposition of Kernel Scores for Generative Models](https://mlsanity.com/paper/2310.05833/) (2023)
- [On Calibration of Large Language Models: From Response To Capability](https://mlsanity.com/paper/2602.13540/) (2026)
- [Bias-variance decompositions: the exclusive privilege of Bregman divergences](https://mlsanity.com/paper/2501.18581/) (2025)
- [Federated Continual Learning Goes Online: Uncertainty-Aware Memory Management for Vision Tasks and Beyond](https://mlsanity.com/paper/2405.18925/) (2024)
- [Uncertainty Quantification for Machine Learning: One Size Does Not Fit All](https://mlsanity.com/paper/2512.12341/) (2025)
- [How to Leverage Predictive Uncertainty Estimates for Reducing Catastrophic Forgetting in Online Continual Learning](https://mlsanity.com/paper/2407.07668/) (2024)
- [Uncertainty Quantification for Regression: A Unified Framework based on kernel scores](https://mlsanity.com/paper/2510.25599/) (2025)
- [Evaluating Epistemic Uncertainty: Beyond OOD Detection and Active Learning](https://mlsanity.com/paper/2607.14817/) (2026)
- [Finer Disentanglement of Aleatoric Uncertainty Can Accelerate Chemical Histopathology Imaging](https://mlsanity.com/paper/2502.20532/) (2025)
- [Uncertainty Quantification with Proper Scoring Rules: Adjusting Measures to Prediction Tasks](https://mlsanity.com/paper/2505.22538/) (2025)
- [Subjective Risk Decomposition: A New View for Uncertainty Quantification](https://mlsanity.com/paper/2607.15196/) (2026)
- [When three experiments are better than two: Avoiding intractable correlated aleatoric uncertainty by leveraging a novel bias--variance tradeoff](https://mlsanity.com/paper/2509.04363/) (2025)

## Abstract (arXiv)

Reliably estimating the uncertainty of a prediction throughout the model lifecycle is crucial in many safety-critical applications. The most common way to measure this uncertainty is via the predicted confidence. While this tends to work well for in-domain samples, these estimates are unreliable under domain drift and restricted to classification. Alternatively, proper scores can be used for most predictive tasks but a bias-variance decomposition for model uncertainty does not exist in the current literature. In this work we introduce a general bias-variance decomposition for proper scores, giving rise to the Bregman Information as the variance term. We discover how exponential families and the classification log-likelihood are special cases and provide novel formulations. Surprisingly, we can express the classification case purely in the logit space. We showcase the practical relevance of this decomposition on several downstream tasks, including model ensembles and confidence regions. Further, we demonstrate how different approximations of the instance-level Bregman Information allow reliable out-of-distribution detection for all degrees of domain drift.