mlsanityData to 14 Oct 2026

A Primer on Bayesian Neural Networks: Review and Debates

arXiv 2309.16314 ↗First posted 28 Sep 2023stat.ML cs.LG math.STcited by 27

Not on the map. Why?

Abstract (arXiv)

Neural networks have achieved remarkable performance across various problem domains, but their widespread applicability is hindered by inherent limitations such as overconfidence in predictions, lack of interpretability, and vulnerability to adversarial attacks. To address these challenges, Bayesian neural networks (BNNs) have emerged as a compelling extension of conventional neural networks, integrating uncertainty estimation into their predictive capabilities. This comprehensive primer presents a systematic introduction to the fundamental concepts of neural networks and Bayesian inference, elucidating their synergistic integration for the development of BNNs. The target audience comprises statisticians with a potential background in Bayesian methods but lacking deep learning expertise, as well as machine learners proficient in deep neural networks but with limited exposure to Bayesian statistics. We provide an overview of commonly employed priors, examining their impact on model behavior and performance. Additionally, we delve into the practical considerations associated with training and inference in BNNs. Furthermore, we explore advanced topics within the realm of BNN research, acknowledging the existence of ongoing debates and controversies. By offering insights into cutting-edge developments, this primer not only equips researchers and practitioners with a solid foundation in BNNs, but also illuminates the potential applications of this dynamic field. As a valuable resource, it fosters an understanding of BNNs and their promising prospects, facilitating further advancements in the pursuit of knowledge and innovation.

Key facts. Cited by 27 papers in the map (data through 14 Oct 2026). Builds on 170 of its 266 references that are in the map. First posted on arXiv 28 Sep 2023.

Builds on

170 of 266 references.

References · 170

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
  5. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,733
  6. 2017On Calibration of Modern Neural Networkscited by 4,960
  7. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,448
  8. 2013Auto-Encoding Variational Bayescited by 4,053
  9. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,926
  10. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,611
  11. 2017Axiomatic Attribution for Deep Networkscited by 3,384
  12. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,621
38 more
  1. 2016Identity Mappings in Deep Residual Networkscited by 1,965
  2. 2017Understanding Black-box Predictions via Influence Functionscited by 1,741
  3. 2016Understanding deep learning requires rethinking generalizationcited by 1,674
  4. 2016Variational Inference: A Review for Statisticianscited by 1,472
  5. 2018ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustnesscited by 1,432
  6. 2020Sharpness-Aware Minimization for Efficiently Improving Generalizationcited by 1,417
  7. 2015DeepFool: a simple and accurate method to fool deep neural networkscited by 1,318
  8. 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,287
  9. 2018Averaging Weights Leads to Wider Optima and Better Generalizationcited by 1,209
  10. 2017Deep Image Priorcited by 1,098
  11. 2018Reconciling modern machine learning practice and the bias-variance trade-offcited by 927
  12. 2020A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challengescited by 819
  13. 2015Optimizing Neural Networks with Kronecker-factored Approximate Curvaturecited by 796
  14. 2016Universal adversarial perturbationscited by 784
  15. 2019Temporal Fusion Transformers for Interpretable Multi-horizon Time Series Forecastingcited by 761
  16. 2019Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descentcited by 738
  17. 2017Spectrally-normalized margin bounds for neural networkscited by 686
  18. 2021A Survey of Uncertainty in Deep Neural Networkscited by 680
  19. 2019Deep Double Descent: Where Bigger Models and More Data Hurtcited by 672
  20. 2013Speech Recognition with Deep Recurrent Neural Networkscited by 669
  21. 2017The Implicit Bias of Gradient Descent on Separable Datacited by 655
  22. 2017Deep Neural Networks as Gaussian Processescited by 622
  23. 2018Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNscited by 588
  24. 2019On Exact Computation with an Infinitely Wide Neural Netcited by 573
  25. 2019A Simple Baseline for Bayesian Uncertainty in Deep Learningcited by 560
  26. 2017Computing Nonvacuous Generalization Bounds for Deep (Stochastic) Neural Networks with Many More Parameters than Training Datacited by 533
  27. 2015Variational Dropout and the Local Reparameterization Trickcited by 490
  28. 2021ConViT: Improving Vision Transformers with Soft Convolutional Inductive Biasescited by 484
  29. 2020Bayesian Deep Learning and a Probabilistic Perspective of Generalizationcited by 480
  30. 2019Measuring Calibration in Deep Learningcited by 446
  31. 2015Deep Kernel Learningcited by 445
  32. 2017Nonparametric regression using deep neural networks with ReLU activation functioncited by 433
  33. 2021Revisiting the Calibration of Modern Neural Networkscited by 390
  34. 2017Training Confidence-calibrated Classifiers for Detecting Out-of-Distribution Samplescited by 389
  35. 2018Pyro: Deep Universal Probabilistic Programmingcited by 380
  36. 2021Laplace Redux -- Effortless Bayesian Deep Learningcited by 365
  37. 2017Snapshot Ensembles: Train 1, get M for freecited by 350
  38. 2017Methods for Interpreting and Understanding Deep Neural Networkscited by 345

50 most cited of 170 All 170 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 9; 2025: 10; 2026: 7.

27 papers.

Sort:

By area:

  1. 2024Position: Bayesian Deep Learning is Needed in the Age of Large-Scale AIcited by 62
  2. 2025Generative Uncertainty in Diffusion Modelscited by 23
  3. 2024Just rephrase it! Uncertainty estimation in closed-source language models via multiple rephrased queriescited by 15
  4. 2024Posterior and variational inference for deep neural networks with heavy-tailed weightscited by 10
  5. 2023Efficient Neural Networks for Tiny Machine Learning: A Comprehensive Reviewcited by 9
  6. 2024Bayesian RG Flow in Neural Network Field Theoriescited by 9
  7. 2024Inferring the Langevin Equation with Uncertainty via Bayesian Neural Networkscited by 4
  8. 2025The Ensemble Kalman Inversion Racecited by 3
  9. 2024Neural Network Prediction of Strong Lensing Systems with Domain Adaptation and Uncertainty Quantificationcited by 2
  10. 2025Posterior Contraction for Sparse Neural Networks in Besov Spaces with Intrinsic Dimensionalitycited by 2
  11. 2024Exploring Diffusion Models' Corruption Stage in Few-Shot Fine-tuning and Mitigating with Bayesian Neural Networkscited by 1
  12. 2025Understanding the Trade-offs in Accuracy and Uncertainty Quantification: Architecture and Inference Choices in Bayesian Neural Networkscited by 1
15 more
  1. 2025Last Layer Hamiltonian Monte Carlocited by 1
  2. 2025Probabilistic Consistency in Machine Learning and Its Connection to Uncertainty Quantificationcited by 1
  3. 2025Bayesian Topological Convolutional Neural Netscited by 1
  4. 2024Just rotate it! Uncertainty estimation in closed-source models via multiple queriescited by 0
  5. 2024Variational Bayesian Bow tie Neural Networks with Shrinkagecited by 0
  6. 2025Label Uncertainty for Ultrasound Segmentationcited by 0
  7. 2025Comparison of Deterministic and Probabilistic Machine Learning Algorithms for Precise Dimensional Control and Uncertainty Quantification in Additive Manufacturingcited by 0
  8. 2025Latent Uncertainty Representations for Video-based Driver Action and Intention Recognitioncited by 0
  9. 2026Minimaxity and Admissibility of Bayesian Neural Networkscited by 0
  10. 2026Uncertainty-aware classification and triage of structural heart disease using electrocardiography and echocardiography metricscited by 0
  11. 2026Soft Specialists: α-R'enyi Ensembles for Uncertainty-Aware LLM Post-Trainingcited by 0
  12. 2026An Auto-Scaling Approach for Serverless Environments Based on a Multi-Expert Consensus Mechanismcited by 0
  13. 2026Uncertainty quantification for trustworthy deep learning: Methods and measurescited by 0
  14. 2026Uncertainty propagation in auto-regressive random neural network modelscited by 0
  15. 2026Sparsifying Stochasticity, Not Capacity: Partial Stochasticity via Deep Weight Factorization of Prior Scalescited by 0