mlsanityData to 14 Oct 2026

If there is no underfitting, there is no Cold Posterior Effect

arXiv 2310.01189 ↗First posted 2 Oct 2023stat.ML cs.LGcited by 2

Not on the map. Why?

Abstract (arXiv)

The cold posterior effect (CPE) (Wenzel et al., 2020) in Bayesian deep learning shows that, for posteriors with a temperature T<1, the resulting posterior predictive could have better performances than the Bayesian posterior (T=1). As the Bayesian posterior is known to be optimal under perfect model specification, many recent works have studied the presence of CPE as a model misspecification problem, arising from the prior and/or from the likelihood function. In this work, we provide a more nuanced understanding of the CPE as we show that misspecification leads to CPE only when the resulting Bayesian posterior underfits. In fact, we theoretically show that if there is no underfitting, there is no CPE.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 19 of its 41 references that are in the map. First posted on arXiv 2 Oct 2023.

Builds on

19 of 41 references.

References · 19

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  3. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,349
  4. 2016Variational Inference: A Review for Statisticianscited by 1,472
  5. 2021What Are Bayesian Neural Network Posteriors Really Like?cited by 308
  6. 2019Cyclical Stochastic Gradient MCMC for Bayesian Deep Learningcited by 203
  7. 2013A General Framework for Updating Belief Distributionscited by 175
  8. 2015On the properties of variational approximations of Gibbs posteriorscited by 121
  9. 2021Bayesian Neural Network Priors Revisitedcited by 121
  10. 2017TensorFlow Distributionscited by 106
  11. 2014Inconsistency of Bayesian Inference for Misspecified Linear Models, and a Proposal for Repairing Itcited by 95
  12. 2016PAC-Bayesian Theory Meets Bayesian Inferencecited by 88
7 more
  1. 2020A statistical theory of cold posteriors in deep neural networkscited by 54
  2. 2022On Uncertainty, Tempering, and Data Augmentation in Bayesian Classificationcited by 46
  3. 2021Data augmentation in Bayesian neural networks and the cold posterior effectcited by 36
  4. 2021Disentangling the Roles of Curation, Data-Augmentation and the Prior in the Cold Posterior Effectcited by 22
  5. 2020PACm-Bayes: Narrowing the Empirical Risk Gap in the Misspecified Bayesian Regimecited by 15
  6. 2022How Tempering Fixes Data Augmentation in Bayesian Neural Networkscited by 13
  7. 2015Manifold Regularized Discriminative Neural Networkscited by 1

Built on by

Cited by year

Citing papers per year: 2024: 1; 2025: 1.

2 papers.

  1. 2024Can a Confident Prior Replace a Cold Posterior?cited by 5
  2. 2025Understanding temperature tuning in energy-based modelscited by 1