# Prior choice affects ability of Bayesian neural networks to identify unknowns

arXiv: https://arxiv.org/abs/2005.04987 · HTML: https://mlsanity.com/paper/2005.04987/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 May 2020.

## Built on by (most cited first, 12 of 12)

- [Priors in Bayesian Deep Learning: A Review](https://mlsanity.com/paper/2105.06868/) (2021)
- [Variational Model Perturbation for Source-Free Domain Adaptation](https://mlsanity.com/paper/2210.10378/) (2022)
- [Explainable Artificial Intelligence for Bayesian Neural Networks: Towards trustworthy predictions of ocean dynamics](https://mlsanity.com/paper/2205.00202/) (2022)
- [A Benchmark on Uncertainty Quantification for Deep Learning Prognostics](https://mlsanity.com/paper/2302.04730/) (2023)
- [BDNNSurv: Bayesian deep neural networks for survival analysis using pseudo values](https://mlsanity.com/paper/2101.03170/) (2021)
- [Bayesian neural networks for the probabilistic forecasting of wind direction and speed using ocean data](https://mlsanity.com/paper/2206.08953/) (2022)
- [From Pointwise to Powerhouse: Initialising Neural Networks with Generative Models](https://mlsanity.com/paper/2310.16695/) (2023)
- [Learning from Uncertain Data: From Possible Worlds to Possible Models](https://mlsanity.com/paper/2405.18549/) (2024)
- [Robust quantum dots charge autotuning using neural network uncertainty](https://mlsanity.com/paper/2406.05175/) (2024)
- [B-PL-PINN: Stabilizing PINN Training with Bayesian Pseudo Labeling](https://mlsanity.com/paper/2507.01714/) (2025)
- [Out-of-Distribution Detection from Small Training Sets using Bayesian Neural Network Classifiers](https://mlsanity.com/paper/2510.06025/) (2025)
- [Uncertainty-quantified Pulse Signal Recovery from Facial Video using Regularized Stochastic Interpolants](https://mlsanity.com/paper/2604.10777/) (2026)

## Abstract (arXiv)

Deep Bayesian neural networks (BNNs) are a powerful tool, though computationally demanding, to perform parameter estimation while jointly estimating uncertainty around predictions. BNNs are typically implemented using arbitrary normal-distributed prior distributions on the model parameters. Here, we explore the effects of different prior distributions on classification tasks in BNNs and evaluate the evidence supporting the predictions based on posterior probabilities approximated by Markov Chain Monte Carlo sampling and by computing Bayes factors. We show that the choice of priors has a substantial impact on the ability of the model to confidently assign data to the correct class (true positive rates). Prior choice also affects significantly the ability of a BNN to identify out-of-distribution instances as unknown (false positive rates). When comparing our results against neural networks (NN) with Monte Carlo dropout we found that BNNs generally outperform NNs. Finally, in our tests we did not find a single best choice as prior distribution. Instead, each dataset yielded the best results under a different prior, indicating that testing alternative options can improve the performance of BNNs.