# Post-hoc loss-calibration for Bayesian neural networks

arXiv: https://arxiv.org/abs/2106.06997 · HTML: https://mlsanity.com/paper/2106.06997/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 42 references that are in the map.
- First posted on arXiv 13 Jun 2021.

## Builds on (most cited first, 19 of 19)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation](https://mlsanity.com/paper/1511.00561/) (2015)
- [Cyclical Learning Rates for Training Neural Networks](https://mlsanity.com/paper/1506.01186/) (2015)
- [Weight Uncertainty in Neural Networks](https://mlsanity.com/paper/1505.05424/) (2015)
- [Variational Dropout and the Local Reparameterization Trick](https://mlsanity.com/paper/1506.02557/) (2015)
- [Stochastic Gradient Hamiltonian Monte Carlo](https://mlsanity.com/paper/1402.4102/) (2014)
- [Probabilistic Backpropagation for Scalable Learning of Bayesian Neural Networks](https://mlsanity.com/paper/1502.05336/) (2015)
- [Cyclical Stochastic Gradient MCMC for Bayesian Deep Learning](https://mlsanity.com/paper/1902.03932/) (2019)
- [A General Framework for Updating Belief Distributions](https://mlsanity.com/paper/1306.6430/) (2013)
- [Expectation Propagation for approximate Bayesian inference](https://mlsanity.com/paper/1301.2294/) (2013)
- [Quality of Uncertainty Quantification for Bayesian Neural Network Inference](https://mlsanity.com/paper/1906.09686/) (2019)
- [Generalized Variational Inference: Three arguments for deriving new Posteriors](https://mlsanity.com/paper/1904.02063/) (2019)
- [Bayesian Dark Knowledge](https://mlsanity.com/paper/1506.04416/) (2015)
- [Stochastic Expectation Propagation](https://mlsanity.com/paper/1506.04132/) (2015)
- [Loss-Calibrated Approximate Inference in Bayesian Neural Networks](https://mlsanity.com/paper/1805.03901/) (2018)
- [Variational Bayesian Decision-making for Continuous Utilities](https://mlsanity.com/paper/1902.00792/) (2019)
- [Generalized Bayesian Posterior Expectation Distillation for Deep Neural Networks](https://mlsanity.com/paper/2005.08110/) (2020)
- [URSABench: Comprehensive Benchmarking of Approximate Bayesian Inference Methods for Deep Neural Networks](https://mlsanity.com/paper/2007.04466/) (2020)
- [Correcting Predictions for Approximate Bayesian Inference](https://mlsanity.com/paper/1909.04919/) (2019)

## Built on by (most cited first, 9 of 9)

- [Amortized Bayesian Experimental Design for Decision-Making](https://mlsanity.com/paper/2411.02064/) (2024)
- [HumBugDB: A Large-scale Acoustic Mosquito Dataset](https://mlsanity.com/paper/2110.07607/) (2021)
- [Audio-based AI classifiers show no evidence of improved COVID-19 screening over simple symptoms checkers](https://mlsanity.com/paper/2212.08570/) (2022)
- [Hallucination Detection and Mitigation in Large Language Models](https://mlsanity.com/paper/2601.09929/) (2026)
- [Long-tailed Classification from a Bayesian-decision-theory Perspective](https://mlsanity.com/paper/2303.06075/) (2023)
- [Amortized Bayesian Decision Making for simulation-based models](https://mlsanity.com/paper/2312.02674/) (2023)
- [Understanding and mitigating difficulties in posterior predictive evaluation](https://mlsanity.com/paper/2405.19747/) (2024)
- [Challenges and Opportunities in Approximate Bayesian Deep Learning for Intelligent IoT Systems](https://mlsanity.com/paper/2112.01675/) (2021)
- [Robust Bayesian Decision Making under Adversarial Uncertainty](https://mlsanity.com/paper/2607.08590/) (2026)

## Abstract (arXiv)

Bayesian decision theory provides an elegant framework for acting optimally under uncertainty when tractable posterior distributions are available. Modern Bayesian models, however, typically involve intractable posteriors that are approximated with, potentially crude, surrogates. This difficulty has engendered loss-calibrated techniques that aim to learn posterior approximations that favor high-utility decisions. In this paper, focusing on Bayesian neural networks, we develop methods for correcting approximate posterior predictive distributions encouraging them to prefer high-utility decisions. In contrast to previous work, our approach is agnostic to the choice of the approximate inference algorithm, allows for efficient test time decision making through amortization, and empirically produces higher quality decisions. We demonstrate the effectiveness of our approach through controlled experiments spanning a diversity of tasks and datasets.