mlsanityData to 14 Oct 2026

Streamlining Prediction in Bayesian Deep Learning

arXiv 2411.18425 ↗First posted 27 Nov 2024cs.LGcited by 5

Not on the map. Why?

Abstract (arXiv)

The rising interest in Bayesian deep learning (BDL) has led to a plethora of methods for estimating the posterior distribution. However, efficient computation of inferences, such as predictions, has been largely overlooked with Monte Carlo integration remaining the standard. In this work we examine streamlining prediction in BDL through a single forward pass without sampling. For this we use local linearisation on activation functions and local Gaussian approximations at linear layers. Thus allowing us to analytically compute an approximation to the posterior predictive distribution. We showcase our approach for both MLP and transformers, such as ViT and GPT-2, and assess its performance on regression and classification tasks. Open-source library: https://github.com/AaltoML/SUQ

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 44 of its 71 references that are in the map. First posted on arXiv 27 Nov 2024.

Builds on

44 of 71 references.

References · 44

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,899
  5. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,448
  6. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,349
  7. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,926
  8. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,218
  9. 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,850
  10. 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,729
  11. 2016Variational Inference: A Review for Statisticianscited by 1,472
  12. 2017Deep Bayesian Active Learning with Image Datacited by 819
32 more
  1. 2017Remote Sensing Image Scene Classification: Benchmark and State of the Artcited by 797
  2. 2015Optimizing Neural Networks with Kronecker-factored Approximate Curvaturecited by 796
  3. 2021MedMNIST v2 -- A large-scale lightweight benchmark for 2D and 3D biomedical image classificationcited by 690
  4. 2019A Simple Baseline for Bayesian Uncertainty in Deep Learningcited by 560
  5. 2020Bayesian Deep Learning and a Probabilistic Perspective of Generalizationcited by 480
  6. 2021Laplace Redux -- Effortless Bayesian Deep Learningcited by 365
  7. 2017Advances in Variational Inferencecited by 236
  8. 2020Being Bayesian, Even Just a Bit, Fixes Overconfidence in ReLU Networkscited by 231
  9. 2020Training independent subnetworks for robust predictioncited by 177
  10. 2022Uncertainty Quantification in Scientific Machine Learning: Methods, Metrics, and Comparisonscited by 170
  11. 2021SOFT: Softmax-free Transformer with Linear Complexitycited by 128
  12. 2021Bayesian Neural Network Priors Revisitedcited by 121
  13. 2023Bayesian Low-rank Adaptation for Large Language Modelscited by 115
  14. 2021Scalable Marginal Likelihood Estimation for Model Selection in Deep Learningcited by 106
  15. 2020Bayesian Deep Learning via Subnetwork Inferencecited by 87
  16. 2020The Case for Bayesian Deep Learningcited by 69
  17. 2024Variational Learning is Effective for Large Deep Networkscited by 65
  18. 2023Prediction-Oriented Bayesian Active Learningcited by 62
  19. 2024Position: Bayesian Deep Learning is Needed in the Age of Large-Scale AIcited by 62
  20. 2019Uncertainty Quantification with Statistical Guarantees in End-to-End Autonomous Driving Controlcited by 61
  21. 2020All You Need is a Good Functional Prior for Bayesian Deep Learningcited by 61
  22. 2021Mixtures of Laplace Approximations for Improved Post-Hoc Uncertainty in Deep Learningcited by 25
  23. 2022Wide Mean-Field Bayesian Neural Networks Ignore the Datacited by 22
  24. 2021Periodic Activation Functions Induce Stationaritycited by 21
  25. 2023Fixing Overconfidence in Dynamic Neural Networkscited by 19
  26. 2023Black Box Variational Inference with a Deterministic Objective: Faster, More Accurate, and Even More Black Boxcited by 17
  27. 2024Uncertainty Quantification via Stable Distribution Propagationcited by 10
  28. 2022Gaussian Pre-Activations in Neural Networks: Myth or Reality?cited by 9
  29. 2024Function-space Parameterization of Neural Networks for Sequential Learningcited by 8
  30. 2020Tractable Approximate Gaussian Inference for Bayesian Neural Networkscited by 7
  31. 2023Efficient Parametric Approximations of Neural Network Function Space Distancecited by 7
  32. 2024Variational Inference Failures Under Model Symmetries: Permutation Invariant Posteriors for Bayesian Neural Networkscited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 2; 2026: 2.

5 papers.

  1. 2024Post-hoc Probabilistic Vision-Language Modelscited by 13
  2. 2025Variational Visual Question Answering for Uncertainty-Aware Selective Predictioncited by 5
  3. 2025Rethinking Approximate Gaussian Inference in Classificationcited by 2
  4. 2026Breaking the Illusion: When Positive Meets Negative in Multimodal Decodingcited by 1
  5. 2026Calibrated Sampling-Free Uncertainty Estimation in Bayesian Deep Learningcited by 0