mlsanityData to 14 Oct 2026

The Sensitivity of Variational Bayesian Neural Network Performance to Hyperparameters

arXiv 2509.20574 ↗First posted 24 Sep 2025cs.LG stat.MLcited by 0

Not on the map. Why?

Abstract (arXiv)

In scientific applications, predictive modeling is often of limited use without accurate uncertainty quantification (UQ) to indicate when a model may be extrapolating or when more data needs to be collected. Bayesian Neural Networks (BNNs) produce predictive uncertainty by propagating uncertainty in neural network (NN) weights and offer the promise of obtaining not only an accurate predictive model but also accurate UQ. However, in practice, obtaining accurate UQ with BNNs is difficult due in part to the approximations used for practical model training and in part to the need to choose a suitable set of hyperparameters; these hyperparameters outnumber those needed for traditional NNs and often have opaque effects on the results. We aim to shed light on the effects of hyperparameter choices for BNNs by performing a global sensitivity analysis of BNN performance under varying hyperparameter settings. Our results indicate that many of the hyperparameters interact with each other to affect both predictive accuracy and UQ. For improved usage of BNNs in real-world applications, we suggest that global sensitivity analysis, or related methods such as Bayesian optimization, should be used to aid in dimensionality reduction and selection of hyperparameters to ensure accurate UQ in BNNs.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 20 of its 41 references that are in the map. First posted on arXiv 24 Sep 2025.

Builds on

20 of 41 references.

References · 20

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  2. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,448
  3. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,926
  4. 2016Variational Inference: A Review for Statisticianscited by 1,472
  5. 2020A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challengescited by 819
  6. 2019A Simple Baseline for Bayesian Uncertainty in Deep Learningcited by 560
  7. 2021Laplace Redux -- Effortless Bayesian Deep Learningcited by 365
  8. 2020Hands-on Bayesian Neural Networks -- a Tutorial for Deep Learning Userscited by 338
  9. 2015Probabilistic Backpropagation for Scalable Learning of Bayesian Neural Networkscited by 295
  10. 2017Multiplicative Normalizing Flows for Variational Bayesian Neural Networkscited by 157
  11. 2016Fast Bayesian Optimization of Machine Learning Hyperparameters on Large Datasetscited by 132
  12. 2020Bayesian Neural Networks: An Introduction and Surveycited by 131
8 more
  1. 2021Bayesian Neural Network Priors Revisitedcited by 121
  2. 2016R'enyi Divergence Variational Inferencecited by 104
  3. 2016Structured and Efficient Variational Deep Learning with Matrix Gaussian Posteriorscited by 82
  4. 2020All You Need is a Good Functional Prior for Bayesian Deep Learningcited by 61
  5. 2019Challenges in Markov chain Monte Carlo for Bayesian neural networkscited by 44
  6. 2021GradDiv: Adversarial Robustness of Randomized Neural Networks via Gradient Diversity Regularizationcited by 34
  7. 2018Quantifying model form uncertainty in Reynolds-averaged turbulence models with Bayesian deep neural networkscited by 20
  8. 2018Quantifying Uncertainty in Discrete-Continuous and Skewed Data with Bayesian Deep Learningcited by 13

Built on by

Not cited yet.