mlsanityData to 7 Oct 2026

Position: Bayesian Deep Learning is Needed in the Age of Large-Scale AI

arXiv 2402.00809 ↗First posted 1 Feb 2024cs.LG stat.MLcited by 60

Not on the map. Why?

Abstract (arXiv)

In the current landscape of deep learning research, there is a predominant emphasis on achieving high predictive accuracy in supervised tasks involving large image and language datasets. However, a broader perspective reveals a multitude of overlooked metrics, tasks, and data types, such as uncertainty, active and continual learning, and scientific data, that demand attention. Bayesian deep learning (BDL) constitutes a promising avenue, offering advantages across these diverse settings. This paper posits that BDL can elevate the capabilities of deep learning. It revisits the strengths of BDL, acknowledges existing challenges, and highlights some exciting research avenues aimed at addressing these obstacles. Looking ahead, the discussion focuses on possible ways to combine large-scale foundation models with BDL to unlock their full potential.

Key facts. Cited by 60 papers in the map (data through 7 Oct 2026). Builds on 159 of its 239 references that are in the map. First posted on arXiv 1 Feb 2024.

Builds on

159 of 239 references.

References · 159

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  3. 2022Flow Matching for Generative Modelingcited by 5,426
  4. 2017On Calibration of Modern Neural Networkscited by 4,860
  5. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  6. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  7. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  8. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,597
  9. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
  10. 2022Language Models (Mostly) Know What They Knowcited by 1,783
  11. 2021An Explanation of In-context Learning as Implicit Bayesian Inferencecited by 915
  12. 2017Deep Bayesian Active Learning with Image Datacited by 812
38 more
  1. 2023Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedbackcited by 743
  2. 2017Deep Neural Networks as Gaussian Processescited by 617
  3. 2019A Simple Baseline for Bayesian Uncertainty in Deep Learningcited by 555
  4. 2015Weight Uncertainty in Neural Networkscited by 554
  5. 2016Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithmcited by 498
  6. 2023Large Language Models Are Zero-Shot Time Series Forecasterscited by 491
  7. 2020Bayesian Deep Learning and a Probabilistic Perspective of Generalizationcited by 480
  8. 2015Deep Kernel Learningcited by 440
  9. 2018Conditional Neural Processescited by 421
  10. 2017A Tutorial on Thompson Samplingcited by 420
  11. 2018Pyro: Deep Universal Probabilistic Programmingcited by 379
  12. 2022Last Layer Re-Training is Sufficient for Robustness to Spurious Correlationscited by 379
  13. 2021Laplace Redux -- Effortless Bayesian Deep Learningcited by 359
  14. 2014Stochastic Gradient Hamiltonian Monte Carlocited by 341
  15. 2021Transformers Can Do Bayesian Inferencecited by 338
  16. 2017Variational Continual Learningcited by 335
  17. 2018Gaussian Process Behaviour in Wide Deep Neural Networkscited by 314
  18. 2021What Are Bayesian Neural Network Posteriors Really Like?cited by 307
  19. 2017Stochastic Gradient Descent as Approximate Bayesian Inferencecited by 298
  20. 2017Variational Dropout Sparsifies Deep Neural Networkscited by 265
  21. 2020Being Bayesian, Even Just a Bit, Fixes Overconfidence in ReLU Networkscited by 227
  22. 2019Cyclical Stochastic Gradient MCMC for Bayesian Deep Learningcited by 203
  23. 2018Bayesian Model-Agnostic Meta-Learningcited by 194
  24. 2019Evaluating Scalable Bayesian Deep Learning Methods for Robust Computer Visioncited by 182
  25. 2023Protein Design with Guided Discrete Diffusioncited by 154
  26. 2019Convolutional Conditional Neural Processescited by 139
  27. 2020Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertaintycited by 128
  28. 2019Functional Variational Bayesian Neural Networkscited by 127
  29. 2021Bayesian Neural Network Priors Revisitedcited by 121
  30. 2023Modern Bayesian Experimental Designcited by 121
  31. 2017Bayesian Compression for Deep Learningcited by 116
  32. 2023Bayesian Low-rank Adaptation for Large Language Modelscited by 112
  33. 2021Scalable Marginal Likelihood Estimation for Model Selection in Deep Learningcited by 105
  34. 2021Repulsive Deep Ensembles are Bayesiancited by 103
  35. 2022Plex: Towards Reliability using Pretrained Large Model Extensionscited by 102
  36. 2021Priors in Bayesian Deep Learning: A Reviewcited by 101
  37. 2022Accelerating Bayesian Optimization for Biological Sequence Design with Denoising Autoencoderscited by 99
  38. 2021The Bayesian Learning Rulecited by 95

50 most cited of 159 All 159 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 17; 2025: 25; 2026: 18.

60 papers.

Sort:

By area:

  1. 2024How to Build the Virtual Cell with Artificial Intelligence: Priorities and Opportunitiescited by 50
  2. 2024On Information-Theoretic Measures of Predictive Uncertaintycited by 16
  3. 2024Post-hoc Probabilistic Vision-Language Modelscited by 13
  4. 2024Linearization Turns Neural Operators into Function-Valued Gaussian Processescited by 11
  5. 2025Microcanonical Langevin Ensembles: Advancing the Sampling of Bayesian Neural Networkscited by 10
  6. 2024Sources of Uncertainty in 3D Scene Reconstructioncited by 9
  7. 2024Bayesian Kolmogorov Arnold Networks (Bayesian_KANs): A Probabilistic Approach to Enhance Accuracy and Interpretabilitycited by 8
  8. 2024Predictive Coding Networks and Inference Learning: Tutorial and Surveycited by 7
  9. 2024Physics-Informed Variational State-Space Gaussian Processescited by 7
  10. 2024Robust and Efficient Fine-tuning of LLMs with Bayesian Reparameterization of Low-Rank Adaptationcited by 6
  11. 2024A Review of Bayesian Uncertainty Quantification in Deep Probabilistic Image Segmentationcited by 6
  12. 2025BLIPs: Bayesian Learned Interatomic Potentialscited by 6
38 more
  1. 2024Streamlining Prediction in Bayesian Deep Learningcited by 5
  2. 2025laplax -- Laplace Approximations with JAXcited by 5
  3. 2025From Global to Local: A Scalable Benchmark for Local Posterior Samplingcited by 5
  4. 2024Bayesian Sheaf Neural Networkscited by 4
  5. 2025SiLVR: Scalable Lidar-Visual Radiance Field Reconstruction with Uncertainty Quantificationcited by 3
  6. 2025Flow Stochastic Segmentation Networkscited by 3
  7. 2026Position: agentic AI orchestration should be Bayes-consistentcited by 3
  8. 2026Uncertainty in Physics and AI: Taxonomy, Quantification, and Validationcited by 3
  9. 2024Thermodynamic Bayesian Inferencecited by 2
  10. 2024Optimization Proxies using Limited Labeled Data and Training Time -- A Semi-Supervised Bayesian Neural Network Approachcited by 2
  11. 2024Why you don't overfit, and don't need Bayes if you only train for one epochcited by 2
  12. 2025Bayesian Predictive Codingcited by 2
  13. 2025From predictions to confidence intervals: an empirical study of conformal prediction methods for in-context learningcited by 2
  14. 2025Learning to Disentangle Latent Reasoning Rules with Language VAEs: A Systematic Studycited by 2
  15. 2026On the Interplay of Priors and Overparametrization in Bayesian Neural Network Posteriorscited by 2
  16. 2024Ex Uno Pluria: Insights on Ensembling in Low Precision Number Systemscited by 1
  17. 2025Explainable Bayesian deep learning through input-skip Latent Binary Bayesian Neural Networkscited by 1
  18. 2025Epistemic Errors of Imperfect Multitask Learners When Distributions Shiftcited by 1
  19. 2025Scaling Laws for Uncertainty in Deep Learningcited by 1
  20. 2025Last Layer Hamiltonian Monte Carlocited by 1
  21. 2025Single- to multi-fidelity history-dependent learning with uncertainty quantification and disentanglement: application to data-driven constitutive modelingcited by 1
  22. 2025Meta-Learning at Scale for Large Language Models via Low-Rank Amortized Bayesian Meta-Learningcited by 1
  23. 2025Bayesian Topological Convolutional Neural Netscited by 1
  24. 2026Linear-LLM-SCM: Benchmarking LLMs for Coefficient Elicitation in Linear-Gaussian Causal Modelscited by 1
  25. 2026The Well-Tempered Classifier: Some Elementary Properties of Temperature Scalingcited by 1
  26. 2026Calibrating Scientific Foundation Models with Inference-Time Stochastic Attentioncited by 1
  27. 2026LLMs as Implicit Imputers: Uncertainty Should Scale with Missing Informationcited by 1
  28. 2026bde: A Python Package for Bayesian Deep Ensembles via MILEcited by 1
  29. 2026Position: The Time for Sampling Is Now! Charting a New Course for Bayesian Deep Learningcited by 1
  30. 2024Variation Due to Regularization Tractably Recovers Bayesian Deep Learningcited by 0
  31. 2025Learning Model Successorscited by 0
  32. 2025Variational Bayesian Pseudo-Coresetcited by 0
  33. 2025Bayesian Deep Learning for Discrete Choicecited by 0
  34. 2025Variational Deep Learning via Implicit Regularizationcited by 0
  35. 2025Engineering Sentiencecited by 0
  36. 2025Uncertainty Quantification in Probabilistic Machine Learning Models: Theory, Methods, and Insightscited by 0
  37. 2025Latent Uncertainty Representations for Video-based Driver Action and Intention Recognitioncited by 0
  38. 2025Partial Trace-Class Bayesian Neural Networkscited by 0

50 most cited of 60 All 60 (JSON)