mlsanityData to 7 Oct 2026

Practical Deep Learning with Bayesian Principles

arXiv 1906.02506 ↗First posted 6 Jun 2019stat.ML cs.LGcited by 158

Not on the map. Why?

Abstract (arXiv)

Bayesian methods promise to fix many shortcomings of deep learning, but they are impractical and rarely match the performance of standard methods, let alone improve them. In this paper, we demonstrate practical training of deep networks with natural-gradient variational inference. By applying techniques such as batch normalisation, data augmentation, and distributed training, we achieve similar performance in about the same number of epochs as the Adam optimiser, even on large datasets such as ImageNet. Importantly, the benefits of Bayesian principles are preserved: predictive probabilities are well-calibrated, uncertainties on out-of-distribution data are improved, and continual-learning performance is boosted. This work enables practical deep learning while preserving benefits of Bayesian principles. A PyTorch implementation is available as a plug-and-play optimiser.

Key facts. Cited by 158 papers in the map (data through 7 Oct 2026). First posted on arXiv 6 Jun 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 40; 2022: 33; 2023: 27; 2024: 24; 2025: 22; 2026: 12.

158 papers.

Sort:

By area:

  1. 2021Generalized Out-of-Distribution Detection: A Surveycited by 796
  2. 2021A Survey of Uncertainty in Deep Neural Networkscited by 675
  3. 2021Laplace Redux -- Effortless Bayesian Deep Learningcited by 359
  4. 2021What Are Bayesian Neural Network Posteriors Really Like?cited by 307
  5. 2022Prioritized Training on Points that are Learnable, Worth Learning, and Not Yet Learntcited by 233
  6. 2021Uncertainty Weighted Actor-Critic for Offline Reinforcement Learningcited by 125
  7. 2021Bayesian Neural Network Priors Revisitedcited by 121
  8. 2021Scalable Marginal Likelihood Estimation for Model Selection in Deep Learningcited by 105
  9. 2021Priors in Bayesian Deep Learning: A Reviewcited by 101
  10. 2021The Bayesian Learning Rulecited by 95
  11. 2021The Promises and Pitfalls of Deep Kernel Learningcited by 81
  12. 2023Model Merging by Uncertainty-Based Gradient Matchingcited by 77
38 more
  1. 2022Personalized Federated Learning via Variational Bayesian Inferencecited by 68
  2. 2022Trustworthy clinical AI solutions: a unified review of uncertainty quantification in deep learning models for medical image analysiscited by 62
  3. 2024Variational Learning is Effective for Large Deep Networkscited by 62
  4. 2022A Simple Approach to Improve Single-Model Deep Uncertainty via Distance-Awarenesscited by 49
  5. 2022Benchmarking Bayesian Deep Learning on Diabetic Retinopathy Detection Taskscited by 49
  6. 2021Natural continual learning: success is a journey, not (just) a destinationcited by 46
  7. 2022On Uncertainty, Tempering, and Data Augmentation in Bayesian Classificationcited by 46
  8. 2022Invariance Learning in Deep Neural Networks with Differentiable Laplace Approximationscited by 42
  9. 2023Tractable Function-Space Variational Inference in Bayesian Neural Networkscited by 41
  10. 2021LiBRe: A Practical Bayesian Approach to Adversarial Detectioncited by 40
  11. 2022SAM as an Optimal Relaxation of Bayescited by 40
  12. 2023A review of uncertainty quantification in medical image analysis: probabilistic and non-probabilistic methodscited by 39
  13. 2021Infinitely Deep Bayesian Neural Networks with Stochastic Differential Equationscited by 38
  14. 2021Data augmentation in Bayesian neural networks and the cold posterior effectcited by 36
  15. 2024BEARS Make Neuro-Symbolic Models Aware of their Reasoning Shortcutscited by 36
  16. 2023Making Substitute Models More Bayesian Can Enhance Transferability of Adversarial Examplescited by 32
  17. 2022Adapting the Linearised Laplace Model Evidence for Modern Deep Learningcited by 29
  18. 2021NoiseGrad: Enhancing Explanations by Introducing Stochasticity to Model Weightscited by 28
  19. 2022Accelerated Linearized Laplace Approximation for Bayesian Deep Learningcited by 27
  20. 2023Uncertainty Quantification in Machine Learning for Engineering Design and Health Prognostics: A Tutorialcited by 27
  21. 2021Bayesian Inference with Certifiable Adversarial Robustnesscited by 26
  22. 2023A Primer on Bayesian Neural Networks: Review and Debatescited by 26
  23. 2021An Effective Baseline for Robustness to Distributional Shiftcited by 25
  24. 2021Mixtures of Laplace Approximations for Improved Post-Hoc Uncertainty in Deep Learningcited by 25
  25. 2021A Survey on Epistemic (Model) Uncertainty in Supervised Learning: Recent Advances and Applicationscited by 24
  26. 2022BayesCap: Bayesian Identity Cap for Calibrated Uncertainty in Frozen Neural Networkscited by 23
  27. 2021Tractable structured natural gradient descent using local parameterizationscited by 22
  28. 2022Wide Mean-Field Bayesian Neural Networks Ignore the Datacited by 22
  29. 2022Bayesian Learning for Neural Networks: an algorithmic surveycited by 22
  30. 2021Learning PAC-Bayes Priors for Probabilistic Neural Networkscited by 21
  31. 2022Generalized Variational Inference in Function Spaces: Gaussian Measures meet Bayesian Deep Learningcited by 20
  32. 2023Beyond Deep Ensembles: A Large-Scale Evaluation of Bayesian Deep Learning under Distribution Shiftcited by 20
  33. 2024Gaussian Stochastic Weight Averaging for Bayesian Low-Rank Adaptation of Large Language Modelscited by 20
  34. 2024A Unified and General Framework for Continual Learningcited by 19
  35. 2021Explaining Bayesian Neural Networkscited by 18
  36. 2023Federated Learning as Variational Inference: A Scalable Expectation Propagation Approachcited by 17
  37. 2021Being a Bit Frequentist Improves Bayesian Neural Networkscited by 15
  38. 2021Efficient Online Bayesian Inference for Neural Banditscited by 15

50 most cited of 158 All 158 (JSON)