mlsanityData to 7 Oct 2026

Variational Bayesian Last Layers

arXiv 2404.11599 ↗First posted 17 Apr 2024cs.LG cs.CV stat.MLcited by 67

Not on the map. Why?

Abstract (arXiv)

We introduce a deterministic variational formulation for training Bayesian last layer neural networks. This yields a sampling-free, single-pass model and loss that effectively improves uncertainty estimation. Our variational Bayesian last layer (VBLL) can be trained and evaluated with only quadratic complexity in last layer width, and is thus (nearly) computationally free to add to standard architectures. We experimentally investigate VBLLs, and show that they improve predictive accuracy, calibration, and out of distribution detection over baselines across both regression and classification. Finally, we investigate combining VBLL layers with variational Bayesian feature learning, yielding a lower variance collapsed variational inference method for Bayesian neural networks.

Key facts. Cited by 67 papers in the map (data through 7 Oct 2026). Builds on 52 of its 86 references that are in the map. First posted on arXiv 17 Apr 2024.

Builds on

52 of 86 references.

References · 52

  1. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  2. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  3. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  4. 2016Wide Residual Networkscited by 3,057
  5. 2017Prototypical Networks for Few-shot Learningcited by 3,001
  6. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,362
  7. 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,274
  8. 2019A Simple Baseline for Bayesian Uncertainty in Deep Learningcited by 555
  9. 2020Bayesian Deep Learning and a Probabilistic Perspective of Generalizationcited by 480
  10. 2019Likelihood Ratios for Out-of-Distribution Detectioncited by 451
  11. 2015Deep Kernel Learningcited by 440
  12. 2017A Tutorial on Thompson Samplingcited by 420
38 more
  1. 2019Your Classifier is Secretly an Energy Based Model and You Should Treat it Like Onecited by 402
  2. 2013Gaussian Processes for Big Datacited by 377
  3. 2021Laplace Redux -- Effortless Bayesian Deep Learningcited by 359
  4. 2021What Are Bayesian Neural Network Posteriors Really Like?cited by 307
  5. 2015Scalable Bayesian Optimization Using Deep Neural Networkscited by 284
  6. 2019Monte Carlo Gradient Estimation in Machine Learningcited by 281
  7. 2020Being Bayesian, Even Just a Bit, Fixes Overconfidence in ReLU Networkscited by 227
  8. 2021A Simple Fix to Mahalanobis Distance for Improving Near-OOD Detectioncited by 206
  9. 2018When Gaussian Process Meets Big Data: A Review of Scalable GPscited by 205
  10. 2020Efficient and Scalable Bayesian Neural Nets with Rank-1 Factorscited by 171
  11. 2019Functional Variational Bayesian Neural Networkscited by 127
  12. 2021Bayesian Neural Network Priors Revisitedcited by 121
  13. 2021Priors in Bayesian Deep Learning: A Reviewcited by 101
  14. 2014Inconsistency of Bayesian Inference for Misspecified Linear Models, and a Proposal for Repairing Itcited by 92
  15. 2016Stochastic Variational Deep Kernel Learningcited by 92
  16. 2020Bayesian Deep Learning via Subnetwork Inferencecited by 85
  17. 2021The Promises and Pitfalls of Deep Kernel Learningcited by 81
  18. 2019'In-Between' Uncertainty in Bayesian Neural Networkscited by 79
  19. 2018Efficient Exploration through Bayesian Deep Q-Networkscited by 67
  20. 2022Do Bayesian Neural Networks Need To Be Fully Stochastic?cited by 63
  21. 2019Generalized Variational Inference: Three arguments for deriving new Posteriorscited by 57
  22. 2020A statistical theory of cold posteriors in deep neural networkscited by 53
  23. 2019Continuous Meta-Learning without Taskscited by 50
  24. 2022A Simple Approach to Improve Single-Model Deep Uncertainty via Distance-Awarenesscited by 49
  25. 2021Shallow Bayesian Meta Learning for Real-World Few-Shot Recognitioncited by 48
  26. 2018Meta-Learning Priors for Efficient Online Bayesian Regressioncited by 47
  27. 2022On Uncertainty, Tempering, and Data Augmentation in Bayesian Classificationcited by 46
  28. 2019Challenges in Markov chain Monte Carlo for Bayesian neural networkscited by 43
  29. 2017Covariances, Robustness, and Variational Bayescited by 39
  30. 2019Radial Bayesian Neural Networks: Beyond Discrete Support In Large-Scale Bayesian Deep Learningcited by 37
  31. 2021Correlated Input-Dependent Label Noise in Large-Scale Image Classificationcited by 37
  32. 2019Benchmarking the Neural Linear Model for Regressioncited by 31
  33. 2020Learnable Uncertainty under Laplace Approximationscited by 29
  34. 2019Expressive Priors in Bayesian Neural Networks: Kernel Combinations and Periodic Functionscited by 22
  35. 2019Variational Bayes under Model Misspecificationcited by 22
  36. 2021Last Layer Marginal Likelihood for Invariance Learningcited by 21
  37. 2023Improved uncertainty quantification for neural networks with Bayesian last layercited by 11
  38. 2021Bayesian Embeddings for Few-Shot Open World Recognitioncited by 8

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 12; 2025: 28; 2026: 27.

67 papers.

Sort:

By area:

  1. 2025Offline Model-Based Optimization: Comprehensive Reviewcited by 14
  2. 2024Scalable Bayesian Learning with posteriorscited by 13
  3. 2024Training-Free Bayesianization for Low-Rank Adapters of Large Language Modelscited by 10
  4. 2024Function Space Diversity for Uncertainty Prediction via Repulsive Last-Layer Ensemblescited by 8
  5. 2024Stochasticity in Motion: An Information-Theoretic Approach to Trajectory Predictioncited by 4
  6. 2024Ranking over Regression for Bayesian Optimization and Molecule Selectioncited by 4
  7. 2025Martingale Posterior Neural Networks for Fast Sequential Decision Makingcited by 4
  8. 2025Variational Digital Twinscited by 4
  9. 2025Position: Biology is the Challenge Physics-Informed ML Needs to Evolvecited by 4
  10. 2026Uncertainty-Aware Reward Modeling for Stable RLHFcited by 4
  11. 2024Lightning UQ Box: A Comprehensive Framework for Uncertainty Quantification in Deep Learningcited by 3
  12. 2025Uncertainty propagation in feed-forward neural network modelscited by 3
38 more
  1. 2025Closed-Form Last Layer Optimizationcited by 3
  2. 2026Richer Bayesian Last Layers with Subsampled NTK Featurescited by 3
  3. 2024Achieving Well-Informed Decision-Making in Drug Discovery: A Comprehensive Calibration Study using Neural Network-Based Structure-Activity Modelscited by 2
  4. 2025From Deep Additive Kernel Learning to Last-Layer Bayesian Neural Networks via Induced Prior Approximationcited by 2
  5. 2025Enhancing Uncertainty Estimation and Interpretability via Bayesian Non-negative Decision Layercited by 2
  6. 2025Toward Efficient Uncertainty in LLMs through Evidential Knowledge Distillationcited by 2
  7. 2025Fine-tuning LLMs with variational Bayesian last layer for high-dimensional Bayesian optimizationcited by 2
  8. 2025CLAPS: Aleatoric-Epistemic Scaling via Last-Layer Laplace for Conformal Regressioncited by 2
  9. 2026Variance-Gated Ensembles: An Epistemic-Aware Framework for Uncertainty Estimationcited by 2
  10. 2026Scalable Variational Bayesian Fine-Tuning of LLMs via Orthogonalized Low-Rank Adapterscited by 2
  11. 2024Exploring Diffusion Models' Corruption Stage in Few-Shot Fine-tuning and Mitigating with Bayesian Neural Networkscited by 1
  12. 2024Flat Posterior Does Matter For Bayesian Model Averagingcited by 1
  13. 2025Last Layer Empirical Bayescited by 1
  14. 2025Optimizing Data Augmentation through Bayesian Model Selectioncited by 1
  15. 2025Revisiting Bayesian Model Averaging in the Era of Foundation Modelscited by 1
  16. 2025Last Layer Hamiltonian Monte Carlocited by 1
  17. 2025Uncertainty-Aware Tabular Prediction: Evaluating VBLL-Enhanced TabPFN in Safety-Critical Medical Datacited by 1
  18. 2025Similarity-Distance-Magnitude Activationscited by 1
  19. 2026Robust Stochastic Gradient Posterior Sampling with Lattice Based Discretisationcited by 1
  20. 2024Multivariate Bayesian Last Layer for Regression with Uncertainty Quantification and Decompositioncited by 0
  21. 2024Uncertainty quantification for White Matter Hyperintensity segmentation detects silent failures and improves automated Fazekas quantificationcited by 0
  22. 2024Bayesian Surrogate Training on Multiple Data Sources: A Hybrid Modeling Strategycited by 0
  23. 2025Variational Bayesian Pseudo-Coresetcited by 0
  24. 2025BotUmc: An Uncertainty-Aware Twitter Bot Detection with Multi-view Causal Inferencecited by 0
  25. 2025On Local Posterior Structure in Deep Ensemblescited by 0
  26. 2025Scalable Deep Basis Kernel Gaussian Processescited by 0
  27. 2025Variational Deep Learning via Implicit Regularizationcited by 0
  28. 2025Reliably Detecting Model Failures in Deployment Without Labelscited by 0
  29. 2025Are Recommenders Self-Aware? Label-Free Recommendation Performance Estimation via Model Uncertaintycited by 0
  30. 2025Multi-modal Bayesian Neural Network Surrogates with Conjugate Last-Layer Estimationcited by 0
  31. 2025Enhancing Safety in Diabetic Retinopathy Detection: Uncertainty-Aware Deep Learning Models with Rejection Capabilitiescited by 0
  32. 2025Latent Uncertainty Representations for Video-based Driver Action and Intention Recognitioncited by 0
  33. 2025Meta-RL with Bayesian Linear Task Modelscited by 0
  34. 2026Active learning for photonic crystalscited by 0
  35. 2026Generalized Information Gathering Under Dynamics Uncertaintycited by 0
  36. 2026Rethinking the Flow-Based Gradual Domain Adaptation: A Semi-Dual Optimal Transport Perspectivecited by 0
  37. 2026Generative Neural Operators through Diffusion Last Layercited by 0
  38. 2026Activation-Space Uncertainty Quantification for Pretrained Networkscited by 0

50 most cited of 67 All 67 (JSON)