# Quality of Uncertainty Quantification for Bayesian Neural Network Inference

arXiv: https://arxiv.org/abs/1906.09686 · HTML: https://mlsanity.com/paper/1906.09686/

## Key facts

- Cited by 59 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Jun 2019.

## Built on by (most cited first, 30 of 59)

- [A Survey of Uncertainty in Deep Neural Networks](https://mlsanity.com/paper/2107.03342/) (2021)
- [What Are Bayesian Neural Network Posteriors Really Like?](https://mlsanity.com/paper/2104.14421/) (2021)
- [CARD: Classification and Regression Diffusion Models](https://mlsanity.com/paper/2206.07275/) (2022)
- [Priors in Bayesian Deep Learning: A Review](https://mlsanity.com/paper/2105.06868/) (2021)
- [Machine learning with data assimilation and uncertainty quantification for dynamical systems: a review](https://mlsanity.com/paper/2303.10462/) (2023)
- [Uncertainty in Natural Language Generation: From Theory to Applications](https://mlsanity.com/paper/2307.15703/) (2023)
- [NeuralUQ: A comprehensive library for uncertainty quantification in neural differential equations and operators](https://mlsanity.com/paper/2208.11866/) (2022)
- [Deep Ensembles from a Bayesian Perspective](https://mlsanity.com/paper/2105.13283/) (2021)
- [Beyond Deep Ensembles: A Large-Scale Evaluation of Bayesian Deep Learning under Distribution Shift](https://mlsanity.com/paper/2306.12306/) (2023)
- [Learning-to-defer for sequential medical decision-making under uncertainty](https://mlsanity.com/paper/2109.06312/) (2021)
- [Uncertainty Quantification by Ensemble Learning for Computational Optical Form Measurements](https://mlsanity.com/paper/2103.01259/) (2021)
- [A Structured Review of Literature on Uncertainty in Machine Learning & Deep Learning](https://mlsanity.com/paper/2406.00332/) (2024)
- [Post-hoc loss-calibration for Bayesian neural networks](https://mlsanity.com/paper/2106.06997/) (2021)
- [Are you using test log-likelihood correctly?](https://mlsanity.com/paper/2212.00219/) (2022)
- [Relaxed Quantile Regression: Prediction Intervals for Asymmetric Noise](https://mlsanity.com/paper/2406.03258/) (2024)
- [Randomized Physics-Informed Neural Networks for Bayesian Data Assimilation](https://mlsanity.com/paper/2407.04617/) (2024)
- [Structure and Distribution Metric for Quantifying the Quality of Uncertainty: Assessing Gaussian Processes, Deep Neural Nets, and Deep Neural Operators for Regression](https://mlsanity.com/paper/2203.04515/) (2022)
- [What is Flagged in Uncertainty Quantification? Latent Density Models for Uncertainty Categorization](https://mlsanity.com/paper/2207.05161/) (2022)
- [Dual Accuracy-Quality-Driven Neural Network for Prediction Interval Generation](https://mlsanity.com/paper/2212.06370/) (2022)
- [Bayesian Autoencoders: Analysing and Fixing the Bernoulli likelihood for Out-of-Distribution Detection](https://mlsanity.com/paper/2107.13304/) (2021)
- [A framework for benchmarking uncertainty in deep regression](https://mlsanity.com/paper/2109.09048/) (2021)
- [Bayesian autoencoders with uncertainty quantification: Towards trustworthy anomaly detection](https://mlsanity.com/paper/2202.12653/) (2022)
- [Multi-fidelity Gaussian process surrogate modeling for regression problems in physics](https://mlsanity.com/paper/2404.11965/) (2024)
- [E-PINNs: Epistemic Physics-Informed Neural Networks](https://mlsanity.com/paper/2503.19333/) (2025)
- [Uncertainty Quantification in PINNs for Turbulent Flows: Bayesian Inference and Repulsive Ensembles](https://mlsanity.com/paper/2604.17156/) (2026)
- [Uncertainty Characteristics Curves: A Systematic Assessment of Prediction Intervals](https://mlsanity.com/paper/2106.00858/) (2021)
- [Benchmarking Bayesian neural networks and evaluation metrics for regression tasks](https://mlsanity.com/paper/2206.06779/) (2022)
- [Uncertainty Calibration in Bayesian Neural Networks via Distance-Aware Priors](https://mlsanity.com/paper/2207.08200/) (2022)
- [Randomized Physics-Informed Machine Learning for Uncertainty Quantification in High-Dimensional Inverse Problems](https://mlsanity.com/paper/2312.06177/) (2023)
- [DistPred: A Distribution-Free Probabilistic Inference Method for Regression and Forecasting](https://mlsanity.com/paper/2406.11397/) (2024)

## Abstract (arXiv)

Bayesian Neural Networks (BNNs) place priors over the parameters in a neural network. Inference in BNNs, however, is difficult; all inference methods for BNNs are approximate. In this work, we empirically compare the quality of predictive uncertainty estimates for 10 common inference methods on both regression and classification tasks. Our experiments demonstrate that commonly used metrics (e.g. test log-likelihood) can be misleading. Our experiments also indicate that inference innovations designed to capture structure in the posterior do not necessarily produce high quality posterior approximations.