# If there is no underfitting, there is no Cold Posterior Effect

arXiv: https://arxiv.org/abs/2310.01189 · HTML: https://mlsanity.com/paper/2310.01189/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 41 references that are in the map.
- First posted on arXiv 2 Oct 2023.

## Builds on (most cited first, 19 of 19)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [What Are Bayesian Neural Network Posteriors Really Like?](https://mlsanity.com/paper/2104.14421/) (2021)
- [Cyclical Stochastic Gradient MCMC for Bayesian Deep Learning](https://mlsanity.com/paper/1902.03932/) (2019)
- [A General Framework for Updating Belief Distributions](https://mlsanity.com/paper/1306.6430/) (2013)
- [On the properties of variational approximations of Gibbs posteriors](https://mlsanity.com/paper/1506.04091/) (2015)
- [Bayesian Neural Network Priors Revisited](https://mlsanity.com/paper/2102.06571/) (2021)
- [TensorFlow Distributions](https://mlsanity.com/paper/1711.10604/) (2017)
- [Inconsistency of Bayesian Inference for Misspecified Linear Models, and a Proposal for Repairing It](https://mlsanity.com/paper/1412.3730/) (2014)
- [PAC-Bayesian Theory Meets Bayesian Inference](https://mlsanity.com/paper/1605.08636/) (2016)
- [A statistical theory of cold posteriors in deep neural networks](https://mlsanity.com/paper/2008.05912/) (2020)
- [On Uncertainty, Tempering, and Data Augmentation in Bayesian Classification](https://mlsanity.com/paper/2203.16481/) (2022)
- [Data augmentation in Bayesian neural networks and the cold posterior effect](https://mlsanity.com/paper/2106.05586/) (2021)
- [Disentangling the Roles of Curation, Data-Augmentation and the Prior in the Cold Posterior Effect](https://mlsanity.com/paper/2106.06596/) (2021)
- [PACm-Bayes: Narrowing the Empirical Risk Gap in the Misspecified Bayesian Regime](https://mlsanity.com/paper/2010.09629/) (2020)
- [How Tempering Fixes Data Augmentation in Bayesian Neural Networks](https://mlsanity.com/paper/2205.13900/) (2022)
- [Manifold Regularized Discriminative Neural Networks](https://mlsanity.com/paper/1511.06328/) (2015)

## Built on by (most cited first, 2 of 2)

- [Can a Confident Prior Replace a Cold Posterior?](https://mlsanity.com/paper/2403.01272/) (2024)
- [Understanding temperature tuning in energy-based models](https://mlsanity.com/paper/2512.09152/) (2025)

## Abstract (arXiv)

The cold posterior effect (CPE) (Wenzel et al., 2020) in Bayesian deep learning shows that, for posteriors with a temperature T<1, the resulting posterior predictive could have better performances than the Bayesian posterior (T=1). As the Bayesian posterior is known to be optimal under perfect model specification, many recent works have studied the presence of CPE as a model misspecification problem, arising from the prior and/or from the likelihood function. In this work, we provide a more nuanced understanding of the CPE as we show that misspecification leads to CPE only when the resulting Bayesian posterior underfits. In fact, we theoretically show that if there is no underfitting, there is no CPE.