# Loss-Calibrated Approximate Inference in Bayesian Neural Networks

arXiv: https://arxiv.org/abs/1805.03901 · HTML: https://mlsanity.com/paper/1805.03901/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 May 2018.

## Built on by (most cited first, 18 of 18)

- [Classifier Calibration: A survey on how to assess and improve predicted class probabilities](https://mlsanity.com/paper/2112.10327/) (2021)
- [Amortized Bayesian Experimental Design for Decision-Making](https://mlsanity.com/paper/2411.02064/) (2024)
- [Control-Aware Prediction Objectives for Autonomous Driving](https://mlsanity.com/paper/2204.13319/) (2022)
- [HumBugDB: A Large-scale Acoustic Mosquito Dataset](https://mlsanity.com/paper/2110.07607/) (2021)
- [Targeted Active Learning for Bayesian Decision-Making](https://mlsanity.com/paper/2106.04193/) (2021)
- [Audio-based AI classifiers show no evidence of improved COVID-19 screening over simple symptoms checkers](https://mlsanity.com/paper/2212.08570/) (2022)
- [Post-hoc loss-calibration for Bayesian neural networks](https://mlsanity.com/paper/2106.06997/) (2021)
- [Approximation-Aware Bayesian Optimization](https://mlsanity.com/paper/2406.04308/) (2024)
- [On Calibrated Model Uncertainty in Deep Learning](https://mlsanity.com/paper/2206.07795/) (2022)
- [Advancing Deep Active Learning & Data Subset Selection: Unifying Principles with Information-Theory Intuitions](https://mlsanity.com/paper/2401.04305/) (2024)
- [Dual Bayesian ResNet: A Deep Learning Approach to Heart Murmur Detection](https://mlsanity.com/paper/2305.16691/) (2023)
- [Loss-calibrated expectation propagation for approximate Bayesian decision-making](https://mlsanity.com/paper/2201.03128/) (2022)
- [On Learning Prediction-Focused Mixtures](https://mlsanity.com/paper/2110.13221/) (2021)
- [Uncertainty Estimation in SARS-CoV-2 B-cell Epitope Prediction for Vaccine Development](https://mlsanity.com/paper/2103.11214/) (2021)
- [Long-tailed Classification from a Bayesian-decision-theory Perspective](https://mlsanity.com/paper/2303.06075/) (2023)
- [Amortized Bayesian Decision Making for simulation-based models](https://mlsanity.com/paper/2312.02674/) (2023)
- [Making Reliable and Flexible Decisions in Long-tailed Classification](https://mlsanity.com/paper/2501.14090/) (2025)
- [Last Layer Hamiltonian Monte Carlo](https://mlsanity.com/paper/2507.08905/) (2025)

## Abstract (arXiv)

Current approaches in approximate inference for Bayesian neural networks minimise the Kullback-Leibler divergence to approximate the true posterior over the weights. However, this approximation is without knowledge of the final application, and therefore cannot guarantee optimal predictions for a given task. To make more suitable task-specific approximations, we introduce a new loss-calibrated evidence lower bound for Bayesian neural networks in the context of supervised learning, informed by Bayesian decision theory. By introducing a lower bound that depends on a utility function, we ensure that our approximation achieves higher utility than traditional methods for applications that have asymmetric utility functions. Furthermore, in using dropout inference, we highlight that our new objective is identical to that of standard dropout neural networks, with an additional utility-dependent penalty term. We demonstrate our new loss-calibrated model with an illustrative medical example and a restricted model capacity experiment, and highlight failure modes of the comparable weighted cross entropy approach. Lastly, we demonstrate the scalability of our method to real world applications with per-pixel semantic segmentation on an autonomous driving data set.