mlsanityData to 7 Oct 2026

Mixtures of Laplace Approximations for Improved Post-Hoc Uncertainty in Deep Learning

arXiv 2111.03577 ↗First posted 5 Nov 2021cs.LG stat.MLcited by 25

Not on the map. Why?

Abstract (arXiv)

Deep neural networks are prone to overconfident predictions on outliers. Bayesian neural networks and deep ensembles have both been shown to mitigate this problem to some extent. In this work, we aim to combine the benefits of the two approaches by proposing to predict with a Gaussian mixture model posterior that consists of a weighted sum of Laplace approximations of independently trained deep neural networks. The method can be used post hoc with any set of pre-trained networks and only requires a small computational and memory overhead compared to regular ensembles. We theoretically validate that our approach mitigates overconfidence "far away" from the training data and empirically compare against state-of-the-art baselines on standard uncertainty quantification benchmarks.

Key facts. Cited by 25 papers in the map (data through 7 Oct 2026). Builds on 44 of its 55 references that are in the map. First posted on arXiv 5 Nov 2021.

Builds on

44 of 55 references.

References · 44

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  4. 2017On Calibration of Modern Neural Networkscited by 4,860
  5. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  6. 2016Wide Residual Networkscited by 3,057
  7. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,857
  8. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,362
  9. 2016Concrete Problems in AI Safetycited by 1,582
  10. 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,274
  11. 2018Deep Anomaly Detection with Outlier Exposurecited by 939
  12. 2015Optimizing Neural Networks with Kronecker-factored Approximate Curvaturecited by 776
32 more
  1. 2014Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Imagescited by 774
  2. 2019A Simple Baseline for Bayesian Uncertainty in Deep Learningcited by 555
  3. 2020Bayesian Deep Learning and a Probabilistic Perspective of Generalizationcited by 480
  4. 2019Deep Ensembles: A Loss Landscape Perspectivecited by 438
  5. 2020Simple and Principled Uncertainty Estimation with Deterministic Deep Learning via Distance Awarenesscited by 389
  6. 2021Laplace Redux -- Effortless Bayesian Deep Learningcited by 359
  7. 2018Why ReLU networks yield high-confidence predictions far away from the training data and how to mitigate the problemcited by 323
  8. 2015Scalable Bayesian Optimization Using Deep Neural Networkscited by 284
  9. 2016Understanding Deep Neural Networks with Rectified Linear Unitscited by 253
  10. 2020Being Bayesian, Even Just a Bit, Fixes Overconfidence in ReLU Networkscited by 227
  11. 2020Pitfalls of In-Domain Uncertainty Estimation and Ensembling in Deep Learningcited by 208
  12. 2019Cyclical Stochastic Gradient MCMC for Bayesian Deep Learningcited by 203
  13. 2020Training independent subnetworks for robust predictioncited by 177
  14. 2020Efficient and Scalable Bayesian Neural Nets with Rank-1 Factorscited by 171
  15. 2018Deep Bayesian Bandits Showdown: An Empirical Comparison of Bayesian Deep Networks for Thompson Samplingcited by 166
  16. 2019Practical Deep Learning with Bayesian Principlescited by 158
  17. 2020Uncertainty Quantification and Deep Ensemblescited by 138
  18. 2018Fast and Scalable Bayesian Deep Learning by Weight-Perturbation in Adamcited by 131
  19. 2021Scalable Marginal Likelihood Estimation for Model Selection in Deep Learningcited by 105
  20. 2017Noisy Natural Gradient as Variational Inferencecited by 94
  21. 2019Approximate Inference Turns Deep Networks into Gaussian Processescited by 89
  22. 2020Bayesian Deep Learning via Subnetwork Inferencecited by 85
  23. 2019'In-Between' Uncertainty in Bayesian Neural Networkscited by 79
  24. 2019BackPACK: Packing more into backpropcited by 77
  25. 2019Bayesian Batch Active Learning as Sparse Subset Approximationcited by 68
  26. 2019A Systematic Comparison of Bayesian Deep Learning Robustness in Diabetic Retinopathy Taskscited by 62
  27. 2020Combining Ensembles and Data Augmentation can Harm your Calibrationcited by 44
  28. 2019Benchmarking the Neural Linear Model for Regressioncited by 31
  29. 2020On Power Laws in Deep Ensemblescited by 26
  30. 2020On Last-Layer Algorithms for Classification: Decoupling Representation from Uncertainty Estimationcited by 23
  31. 2020Mean-Field Approximation to Gaussian-Softmax Integral with Application to Uncertainty Estimationcited by 16
  32. 2020Improving predictions of Bayesian neural nets via local linearizationcited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 6; 2023: 5; 2024: 3; 2025: 9; 2026: 2.

25 papers.

Sort:

By area:

  1. 2022A Simple Approach to Improve Single-Model Deep Uncertainty via Distance-Awarenesscited by 49
  2. 2022A Framework and Benchmark for Deep Batch Active Learning for Regressioncited by 47
  3. 2023Quantification of Uncertainty with Adversarial Modelscited by 28
  4. 2022Accelerated Linearized Laplace Approximation for Bayesian Deep Learningcited by 27
  5. 2022BayesCap: Bayesian Identity Cap for Calibrated Uncertainty in Frozen Neural Networkscited by 23
  6. 2023Towards Efficient MCMC Sampling in Bayesian Neural Networks by Exploiting Symmetrycited by 20
  7. 2022Sampling-based inference for large linear models, with application to linearised Laplacecited by 18
  8. 2022Posterior Refinement Improves Sample Efficiency in Bayesian Neural Networkscited by 14
  9. 2023Riemannian Laplace approximations for Bayesian neural networkscited by 14
  10. 2025Uncertainty Quantification with the Empirical Neural Tangent Kernelcited by 11
  11. 2023Promises and Pitfalls of the Linearized Laplace in Bayesian Optimizationcited by 10
  12. 2023On the Out of Distribution Robustness of Foundation Models in Medical Image Segmentationcited by 6
13 more
  1. 2024Streamlining Prediction in Bayesian Deep Learningcited by 5
  2. 2025laplax -- Laplace Approximations with JAXcited by 5
  3. 2024EventSleep: Sleep Activity Recognition with Event Camerascited by 3
  4. 2025FedBEns: One-Shot Federated Learning based on Bayesian Ensemblecited by 3
  5. 2025How does Bayesian Sampling help Membership Inference Attacks?cited by 2
  6. 2026Scalable Variational Bayesian Fine-Tuning of LLMs via Orthogonalized Low-Rank Adapterscited by 2
  7. 2024Flat Posterior Does Matter For Bayesian Model Averagingcited by 1
  8. 2025Walking on the Fiber: A Simple Geometric Approximation for Bayesian Neural Networkscited by 1
  9. 2025On Local Posterior Structure in Deep Ensemblescited by 0
  10. 2025Are you SURE? Enhancing Multimodal Pretraining with Missing Modalities through Uncertainty Estimationcited by 0
  11. 2025Scalable Bayesian Monte Carlo: fast uncertainty estimation beyond deep ensemblescited by 0
  12. 2025Uncertainty-driven Embedding Convolutioncited by 0
  13. 2026Variational meta-learning inference for low dimensional neural system identificationcited by 0