mlsanityData to 7 Oct 2026

A Simple Baseline for Bayesian Uncertainty in Deep Learning

arXiv 1902.02476 ↗First posted 7 Feb 2019cs.LG cs.AI cs.CVcited by 555

Not on the map. Why?

Abstract (arXiv)

We propose SWA-Gaussian (SWAG), a simple, scalable, and general purpose approach for uncertainty representation and calibration in deep learning. Stochastic Weight Averaging (SWA), which computes the first moment of stochastic gradient descent (SGD) iterates with a modified learning rate schedule, has recently been shown to improve generalization in deep learning. With SWAG, we fit a Gaussian using the SWA solution as the first moment and a low rank plus diagonal covariance also derived from the SGD iterates, forming an approximate posterior distribution over neural network weights; we then sample from this Gaussian distribution to perform Bayesian model averaging. We empirically find that SWAG approximates the shape of the true posterior, in accordance with results describing the stationary distribution of SGD iterates. Moreover, we demonstrate that SWAG performs well on a wide variety of tasks, including out of sample detection, calibration, and transfer learning, in comparison to many popular alternatives including MC dropout, KFAC Laplace, SGLD, and temperature scaling.

Key facts. Cited by 555 papers in the map (data through 7 Oct 2026). First posted on arXiv 7 Feb 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 2; 2021: 90; 2022: 110; 2023: 111; 2024: 86; 2025: 92; 2026: 64.

555 papers.

Sort:

By area:

  1. 2021Generalized Out-of-Distribution Detection: A Surveycited by 796
  2. 2021A Survey of Uncertainty in Deep Neural Networkscited by 675
  3. 2022Git Re-Basin: Merging Models modulo Permutation Symmetriescited by 460
  4. 2021Laplace Redux -- Effortless Bayesian Deep Learningcited by 359
  5. 2021What Are Bayesian Neural Network Posteriors Really Like?cited by 307
  6. 2021In Defense of Pseudo-Labeling: An Uncertainty-Aware Pseudo-label Selection Framework for Semi-Supervised Learningcited by 301
  7. 2021Deep Deterministic Uncertainty: A Simple Baselinecited by 200
  8. 2021DICE: Leveraging Sparsification for Out-of-Distribution Detectioncited by 166
  9. 2022Diverse Weight Averaging for Out-of-Distribution Generalizationcited by 148
  10. 2023Non-Parametric Outlier Synthesiscited by 129
  11. 2023Decomposing Uncertainty for Large Language Models through Input Clarification Ensemblingcited by 113
  12. 2021Scalable Marginal Likelihood Estimation for Model Selection in Deep Learningcited by 105
38 more
  1. 2021DEUP: Direct Epistemic Uncertainty Predictioncited by 102
  2. 2021The Bayesian Learning Rulecited by 95
  3. 2023How Does Information Bottleneck Help Deep Learning?cited by 92
  4. 2023Deep Model Fusion: A Surveycited by 89
  5. 2024Model Stock: All we need is just a few fine-tuned modelscited by 87
  6. 2021Learning Neural Network Subspacescited by 82
  7. 2023Provable Dynamic Fusion for Low-Quality Multimodal Datacited by 81
  8. 2021Stochastic Neural Radiance Fields: Quantifying Uncertainty in Implicit 3D Representationscited by 80
  9. 2021Graph Posterior Network: Bayesian Predictive Uncertainty for Node Classificationcited by 79
  10. 2023Dream the Impossible: Outlier Imagination with Diffusion Modelscited by 77
  11. 2021Loss Surface Simplexes for Mode Connecting Volumes and Fast Ensemblingcited by 75
  12. 2021Fair Conformal Predictors for Applications in Medical Imagingcited by 74
  13. 2023Machine learning with data assimilation and uncertainty quantification for dynamical systems: a reviewcited by 74
  14. 2020Safe Imitation Learning via Fast Bayesian Reward Inference from Preferencescited by 73
  15. 2023Calibration in Deep Learning: A Survey of the State-of-the-Artcited by 72
  16. 2022Personalized Federated Learning via Variational Bayesian Inferencecited by 68
  17. 2024Variational Bayesian Last Layerscited by 67
  18. 2022Better Uncertainty Calibration via Proper Scores for Classification and Beyondcited by 65
  19. 2022Deep Ensembles Work, But Are They Necessary?cited by 63
  20. 2022Do Bayesian Neural Networks Need To Be Fully Stochastic?cited by 63
  21. 2022Training Uncertainty-Aware Classifiers with Conformalized Deep Learningcited by 62
  22. 2022Trustworthy clinical AI solutions: a unified review of uncertainty quantification in deep learning models for medical image analysiscited by 62
  23. 2024Variational Learning is Effective for Large Deep Networkscited by 62
  24. 2024Position: Bayesian Deep Learning is Needed in the Age of Large-Scale AIcited by 60
  25. 2023A Review of Uncertainty Estimation and its Application in Medical Imagingcited by 59
  26. 2021Semantics for Robotic Mapping, Perception and Interaction: A Surveycited by 56
  27. 2022Constrained Policy Optimization via Bayesian World Modelscited by 55
  28. 2022Assaying Out-Of-Distribution Generalization in Transfer Learningcited by 53
  29. 2021Revisiting Design Choices in Offline Model-Based Reinforcement Learningcited by 50
  30. 2022Packed-Ensembles for Efficient Uncertainty Estimationcited by 49
  31. 2021DICE: Diversity in Deep Ensembles via Conditional Redundancy Adversarial Estimationcited by 48
  32. 2021Quantifying Uncertainty in Deep Spatiotemporal Forecastingcited by 48
  33. 2024BLoB: Bayesian Low-Rank Adaptation by Backpropagation for Large Language Modelscited by 48
  34. 2021A Survey on Ensemble Learning under the Era of Deep Learningcited by 47
  35. 2022A Framework and Benchmark for Deep Batch Active Learning for Regressioncited by 47
  36. 2022On Uncertainty, Tempering, and Data Augmentation in Bayesian Classificationcited by 46
  37. 2022Uncertainty-guided Source-free Domain Adaptationcited by 46
  38. 2022Investigating the Impact of Model Misspecification in Neural Simulation-based Inferencecited by 44

50 most cited of 555 All 555 (JSON)