# PACm-Bayes: Narrowing the Empirical Risk Gap in the Misspecified Bayesian Regime

arXiv: https://arxiv.org/abs/2010.09629 · HTML: https://mlsanity.com/paper/2010.09629/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Oct 2020.

## Built on by (most cited first, 15 of 15)

- [Priors in Bayesian Deep Learning: A Review](https://mlsanity.com/paper/2105.06868/) (2021)
- [Bayesian Model Selection, the Marginal Likelihood, and Generalization](https://mlsanity.com/paper/2202.11678/) (2022)
- [Dangers of Bayesian Model Averaging under Covariate Shift](https://mlsanity.com/paper/2106.11905/) (2021)
- [Quantum Conformal Prediction for Reliable Uncertainty Quantification in Quantum Machine Learning](https://mlsanity.com/paper/2304.03398/) (2023)
- [Robust Bayesian Learning for Reliable Wireless AI: Framework and Applications](https://mlsanity.com/paper/2207.00300/) (2022)
- [Calibrating Bayesian Learning via Regularization, Confidence Minimization, and Selective Inference](https://mlsanity.com/paper/2404.11350/) (2024)
- [Robust PACm: Training Ensemble Models Under Misspecification and Outliers](https://mlsanity.com/paper/2203.01859/) (2022)
- [Cold Posteriors through PAC-Bayes](https://mlsanity.com/paper/2206.11173/) (2022)
- [On the Performance of Direct Loss Minimization for Bayesian Neural Networks](https://mlsanity.com/paper/2211.08393/) (2022)
- [The fine print on tempered posteriors](https://mlsanity.com/paper/2309.05292/) (2023)
- [Predictive variational inference: Learn the predictively optimal posterior distribution](https://mlsanity.com/paper/2410.14843/) (2024)
- [Uncertainty estimation under model misspecification in neural network regression](https://mlsanity.com/paper/2111.11763/) (2021)
- [If there is no underfitting, there is no Cold Posterior Effect](https://mlsanity.com/paper/2310.01189/) (2023)
- [Variational Learning of Fractional Posteriors](https://mlsanity.com/paper/2603.27488/) (2026)
- [Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding](https://mlsanity.com/paper/2605.08446/) (2026)

## Abstract (arXiv)

The Bayesian posterior minimizes the "inferential risk" which itself bounds the "predictive risk". This bound is tight when the likelihood and prior are well-specified. However since misspecification induces a gap, the Bayesian posterior predictive distribution may have poor generalization performance. This work develops a multi-sample loss (PACm) which can close the gap by spanning a trade-off between the two risks. The loss is computationally favorable and offers PAC generalization guarantees. Empirical study demonstrates improvement to the predictive distribution.