mlsanityData to 7 Oct 2026

'In-Between' Uncertainty in Bayesian Neural Networks

arXiv 1906.11537 ↗First posted 27 Jun 2019stat.ML cs.AI cs.LGcited by 79

Not on the map. Why?

Abstract (arXiv)

We describe a limitation in the expressiveness of the predictive uncertainty estimate given by mean-field variational inference (MFVI), a popular approximate inference method for Bayesian neural networks. In particular, MFVI fails to give calibrated uncertainty estimates in between separated regions of observations. This can lead to catastrophically overconfident predictions when testing on out-of-distribution data. Avoiding such overconfidence is critical for active learning, Bayesian optimisation and out-of-distribution robustness. We instead find that a classical technique, the linearised Laplace approximation, can handle 'in-between' uncertainty much better for small network architectures.

Key facts. Cited by 79 papers in the map (data through 7 Oct 2026). First posted on arXiv 27 Jun 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 17; 2022: 13; 2023: 16; 2024: 14; 2025: 7; 2026: 12.

79 papers.

Sort:

By area:

  1. 2021Generalized Out-of-Distribution Detection: A Surveycited by 796
  2. 2021A Survey of Uncertainty in Deep Neural Networkscited by 675
  3. 2021Laplace Redux -- Effortless Bayesian Deep Learningcited by 359
  4. 2023Bayesian Low-rank Adaptation for Large Language Modelscited by 112
  5. 2021Scalable Marginal Likelihood Estimation for Model Selection in Deep Learningcited by 105
  6. 2024Variational Bayesian Last Layerscited by 67
  7. 2022Do Bayesian Neural Networks Need To Be Fully Stochastic?cited by 63
  8. 2024Variational Learning is Effective for Large Deep Networkscited by 62
  9. 2022Invariance Learning in Deep Neural Networks with Differentiable Laplace Approximationscited by 42
  10. 2023Tractable Function-Space Variational Inference in Bayesian Neural Networkscited by 41
  11. 2024Bayesian Reward Models for LLM Alignmentcited by 35
  12. 2023BayesDiff: Estimating Pixel-wise Uncertainty in Diffusion via Bayesian Inferencecited by 30
38 more
  1. 2022Adapting the Linearised Laplace Model Evidence for Modern Deep Learningcited by 29
  2. 2022Accelerated Linearized Laplace Approximation for Bayesian Deep Learningcited by 27
  3. 2022NeuralEF: Deconstructing Kernels by Deep Neural Networkscited by 26
  4. 2021Mixtures of Laplace Approximations for Improved Post-Hoc Uncertainty in Deep Learningcited by 25
  5. 2021Transformer Uncertainty Estimation with Hierarchical Stochastic Attentioncited by 21
  6. 2023Beyond Deep Ensembles: A Large-Scale Evaluation of Bayesian Deep Learning under Distribution Shiftcited by 20
  7. 2023Variational Linearized Laplace Approximation for Bayesian Deep Learningcited by 17
  8. 2021NOMU: Neural Optimization-based Model Uncertaintycited by 16
  9. 2021Sparse Uncertainty Representation in Deep Learning with Inducing Weightscited by 16
  10. 2023Improving Neural Additive Models with Bayesian Principlescited by 15
  11. 2022Posterior Refinement Improves Sample Efficiency in Bayesian Neural Networkscited by 14
  12. 2023Riemannian Laplace approximations for Bayesian neural networkscited by 14
  13. 2024Scalable Bayesian Learning with posteriorscited by 13
  14. 2021Trust Your Robots! Predictive Uncertainty Estimation of Neural Networks with Sparse Gaussian Processescited by 12
  15. 2021The Peril of Popular Deep Learning Uncertainty Estimation Methodscited by 11
  16. 2024Linearization Turns Neural Operators into Function-Valued Gaussian Processescited by 11
  17. 2024FSP-Laplace: Function-Space Priors for the Laplace Approximation in Bayesian Deep Learningcited by 11
  18. 2024Manifold Sampling for Differentiable Uncertainty in Radiance Fieldscited by 11
  19. 2025Uncertainty Quantification with the Empirical Neural Tangent Kernelcited by 11
  20. 2023Drug Discovery under Covariate Shift with Domain-Informed Prior Distributions over Functionscited by 10
  21. 2021Wide Mean-Field Variational Bayesian Neural Networks Ignore the Datacited by 7
  22. 2021Bayesian Neural Networks: Essentialscited by 7
  23. 2022Scale-invariant Bayesian Neural Networks with Connectivity Tangent Kernelcited by 7
  24. 2023Discretization-Induced Dirichlet Posterior for Robust Uncertainty Quantification on Regressioncited by 7
  25. 2023Riemannian Laplace Approximation with the Fisher Metriccited by 7
  26. 2023Deep Anti-Regularized Ensembles provide reliable out-of-distribution uncertainty quantificationcited by 6
  27. 2023Implicit Variational Inference for High-Dimensional Posteriorscited by 5
  28. 2021A Novel Regression Loss for Non-Parametric Uncertainty Optimizationcited by 4
  29. 2023Density Uncertainty Layers for Reliable Uncertainty Estimationcited by 4
  30. 2024FedSI: Federated Subnetwork Inference for Efficient Uncertainty Quantificationcited by 4
  31. 2025Martingale Posterior Neural Networks for Fast Sequential Decision Makingcited by 4
  32. 2022Robust uncertainty estimates with out-of-distribution pseudo-inputs trainingcited by 3
  33. 2022Uncertainty Calibration in Bayesian Neural Networks via Distance-Aware Priorscited by 3
  34. 2024A Rate-Distortion View of Uncertainty Quantificationcited by 3
  35. 2024Bayes without Underfitting: Fully Correlated Deep Learning Posteriors via Alternating Projectionscited by 3
  36. 2026Diffusion Epistemic Uncertainty with Asymmetric Learning for Diffusion-Generated Image Detectioncited by 3
  37. 2021Depth Uncertainty Networks for Active Learningcited by 2
  38. 2022Challenges and Pitfalls of Bayesian Unlearningcited by 2

50 most cited of 79 All 79 (JSON)