# Reproducible Parameter Inference Using Bagged Posteriors

arXiv: https://arxiv.org/abs/2311.02019 · HTML: https://mlsanity.com/paper/2311.02019/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 51 references that are in the map.
- First posted on arXiv 3 Nov 2023.

## Builds on (most cited first, 11 of 11)

- [A General Framework for Updating Belief Distributions](https://mlsanity.com/paper/1306.6430/) (2013)
- [Inconsistency of Bayesian Inference for Misspecified Linear Models, and a Proposal for Repairing It](https://mlsanity.com/paper/1412.3730/) (2014)
- [Assigning a value to a power likelihood in a general Bayesian model](https://mlsanity.com/paper/1701.08515/) (2017)
- [Sparsity information and regularization in the horseshoe and other shrinkage priors](https://mlsanity.com/paper/1707.01694/) (2017)
- [General Bayesian Updating and the Loss-Likelihood Bootstrap](https://mlsanity.com/paper/1709.07616/) (2017)
- [Robust Bayesian inference via coarsening](https://mlsanity.com/paper/1506.06101/) (2015)
- [Calibrating general posterior credible regions](https://mlsanity.com/paper/1509.00922/) (2015)
- [Nonparametric learning from Bayesian models with randomized objective functions](https://mlsanity.com/paper/1806.11544/) (2018)
- [Safe-Bayesian Generalized Linear Regression](https://mlsanity.com/paper/1910.09227/) (2019)
- [Reproducible Model Selection Using Bagged Posteriors](https://mlsanity.com/paper/2007.14845/) (2020)
- [Discussion of Big Bayes Stories and BayesBag](https://mlsanity.com/paper/1405.4977/) (2014)

## Built on by (most cited first, 2 of 2)

- [Accurate Large-sample Uncertainty Quantification using Stochastic Gradient Markov Chain Monte Carlo](https://mlsanity.com/paper/2606.00293/) (2026)
- [Large-scale Uncertainty Quantification for Latent Variable Models Using Subsampling Markov Chain Monte Carlo](https://mlsanity.com/paper/2606.00309/) (2026)

## Abstract (arXiv)

Under model misspecification, it is known that Bayesian posteriors often do not properly quantify uncertainty about true or pseudo-true parameters. Even more fundamentally, misspecification leads to a lack of reproducibility in the sense that the same model will yield contradictory posteriors on independent data sets from the true distribution. To define a criterion for reproducible uncertainty quantification under misspecification, we consider the probability that two confidence sets constructed from independent data sets have nonempty overlap, and we establish a lower bound on this overlap probability that holds for any valid confidence sets. We prove that credible sets from the standard posterior can strongly violate this bound, particularly in high-dimensional settings (i.e., with dimension increasing with sample size), indicating that it is not internally coherent under misspecification. To improve reproducibility in an easy-to-use and widely applicable way, we propose to apply bagging to the Bayesian posterior ("BayesBag"'); that is, to use the average of posterior distributions conditioned on bootstrapped datasets. We motivate BayesBag from first principles based on Jeffrey conditionalization and show that the bagged posterior typically satisfies the overlap lower bound. Further, we prove a Bernstein--Von Mises theorem for the bagged posterior, establishing its asymptotic normal distribution. We demonstrate the benefits of BayesBag via simulation experiments and an application to crime rate prediction.