mlsanityData to 7 Oct 2026

A statistical theory of cold posteriors in deep neural networks

arXiv 2008.05912 ↗First posted 13 Aug 2020stat.ML cs.LGcited by 53

Not on the map. Why?

Abstract (arXiv)

To get Bayesian neural networks to perform comparably to standard neural networks it is usually necessary to artificially reduce uncertainty using a "tempered" or "cold" posterior. This is extremely concerning: if the prior is accurate, Bayes inference/decision theory is optimal, and any artificial changes to the posterior should harm performance. While this suggests that the prior may be at fault, here we argue that in fact, BNNs for image classification use the wrong likelihood. In particular, standard image benchmark datasets such as CIFAR-10 are carefully curated. We develop a generative model describing curation which gives a principled Bayesian account of cold posteriors, because the likelihood under this new generative model closely matches the tempered likelihoods used in past work.

Key facts. Cited by 53 papers in the map (data through 7 Oct 2026). First posted on arXiv 13 Aug 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 16; 2022: 11; 2023: 5; 2024: 10; 2025: 8; 2026: 3.

53 papers.

Sort:

By area:

  1. 2021What Are Bayesian Neural Network Posteriors Really Like?cited by 307
  2. 2021Bayesian Neural Network Priors Revisitedcited by 121
  3. 2024Variational Bayesian Last Layerscited by 67
  4. 2024Position: Bayesian Deep Learning is Needed in the Age of Large-Scale AIcited by 60
  5. 2022On Uncertainty, Tempering, and Data Augmentation in Bayesian Classificationcited by 46
  6. 2021Data augmentation in Bayesian neural networks and the cold posterior effectcited by 36
  7. 2023A Primer on Bayesian Neural Networks: Review and Debatescited by 26
  8. 2021Disentangling the Roles of Curation, Data-Augmentation and the Prior in the Cold Posterior Effectcited by 22
  9. 2023Uncertainty-aware Surrogate Models for Airfoil Flow Simulations with Denoising Diffusion Probabilistic Modelscited by 22
  10. 2021Periodic Activation Functions Induce Stationaritycited by 21
  11. 2022UncertaINR: Uncertainty Quantification of End-to-End Implicit Neural Representations for Computed Tomographycited by 19
  12. 2022Theoretical characterization of uncertainty in high-dimensional linear classificationcited by 18
38 more
  1. 2021Bayesian OOD detection with aleatoric uncertainty and outlier exposurecited by 16
  2. 2022How Tempering Fixes Data Augmentation in Bayesian Neural Networkscited by 13
  3. 2024Scalable Bayesian Learning with posteriorscited by 13
  4. 2021Polygonal Unadjusted Langevin Algorithms: Creating stable and efficient adaptive algorithms for neural networkscited by 11
  5. 2021Model Selection for Bayesian Autoencoderscited by 10
  6. 2021Non-asymptotic estimates for TUSLA algorithm for non-convex learning with applications to neural networks with ReLU activation functioncited by 10
  7. 2022Monotonicity and Double Descent in Uncertainty Estimation with Gaussian Processescited by 10
  8. 2024Making Better Use of Unlabelled Data in Bayesian Active Learningcited by 10
  9. 2022Interacting Contour Stochastic Gradient Langevin Dynamicscited by 9
  10. 2021Deep Classifiers with Label Noise Modeling and Distance Awarenesscited by 8
  11. 2025Textual Bayes: Quantifying Prompt Uncertainty in LLM-Based Systemscited by 8
  12. 2022Non-reversible Parallel Tempering for Deep Posterior Approximationcited by 7
  13. 2021Bayesian neural network unit priors and generalized Weibull-tail propertycited by 5
  14. 2022Robustness to corruption in pre-trained Bayesian neural networkscited by 5
  15. 2024Can a Confident Prior Replace a Cold Posterior?cited by 5
  16. 2021How Infinitely Wide Neural Networks Can Benefit from Multi-task Learning -- an Exact Macroscopic Characterizationcited by 4
  17. 2021Posterior Temperature Optimization in Variational Inference for Inverse Problemscited by 3
  18. 2022Posterior temperature optimized Bayesian models for inverse problems in medical imagingcited by 3
  19. 2022Cold Posteriors through PAC-Bayescited by 3
  20. 2023The fine print on tempered posteriorscited by 3
  21. 2024On Uncertainty Quantification for Near-Bayes Optimal Algorithmscited by 3
  22. 2026Uncertainty Quantification in PINNs for Turbulent Flows: Bayesian Inference and Repulsive Ensemblescited by 3
  23. 2023If there is no underfitting, there is no Cold Posterior Effectcited by 2
  24. 2025Learning Hyperparameters via a Data-Emphasized Variational Objectivecited by 2
  25. 2025Are vision language models robust to uncertain inputs?cited by 2
  26. 2026Uncertainty Decomposition via Cyclical SG-MCMC and Soft-label Learning for Subjective NLPcited by 2
  27. 2021Variational Laplace for Bayesian neural networkscited by 1
  28. 2022Identify ambiguous tasks combining crowdsourced labels by weighting Areas Under the Margincited by 1
  29. 2024Temperature Optimization for Bayesian Deep Learningcited by 1
  30. 2025Tempering the Bayes Filter towards Improved Model-Based Estimationcited by 1
  31. 2021Dense Uncertainty Estimation via an Ensemble-based Conditional Latent Variable Modelcited by 0
  32. 2021Weight Pruning and Uncertainty in Radio Galaxy Classificationcited by 0
  33. 2023Function-Space Regularization for Deep Bayesian Classificationcited by 0
  34. 2024Addressing the Inconsistency in Bayesian Deep Learning via Generalized Laplace Approximationcited by 0
  35. 2024On Uniform, Bayesian, and PAC-Bayesian Deep Ensemblescited by 0
  36. 2024Diffusion Priors for Variational Likelihood Estimation and Image Denoisingcited by 0
  37. 2025A weighted-likelihood framework for class imbalance in Bayesian prediction modelscited by 0
  38. 2025Humble your Overconfident Networks: Unlearning Overfitting via Sequential Monte Carlo Tempered Deep Ensemblescited by 0

50 most cited of 53 All 53 (JSON)