# Predictive variational inference: Learn the predictively optimal posterior distribution

arXiv: https://arxiv.org/abs/2410.14843 · HTML: https://mlsanity.com/paper/2410.14843/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 74 references that are in the map.
- First posted on arXiv 18 Oct 2024.

## Builds on (most cited first, 24 of 24)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [Normalizing Flows for Probabilistic Modeling and Inference](https://mlsanity.com/paper/1912.02762/) (2019)
- [Neural Spline Flows](https://mlsanity.com/paper/1906.04032/) (2019)
- [The frontier of simulation-based inference](https://mlsanity.com/paper/1911.01429/) (2019)
- [A General Framework for Updating Belief Distributions](https://mlsanity.com/paper/1306.6430/) (2013)
- [On the properties of variational approximations of Gibbs posteriors](https://mlsanity.com/paper/1506.04091/) (2015)
- [Inconsistency of Bayesian Inference for Misspecified Linear Models, and a Proposal for Repairing It](https://mlsanity.com/paper/1412.3730/) (2014)
- [Robust Generalised Bayesian Inference for Intractable Likelihoods](https://mlsanity.com/paper/2104.07359/) (2021)
- [Bayesian Workflow](https://mlsanity.com/paper/2011.01808/) (2020)
- [Martingale posterior distributions](https://mlsanity.com/paper/2103.15671/) (2021)
- [Investigating the Impact of Model Misspecification in Neural Simulation-based Inference](https://mlsanity.com/paper/2209.01845/) (2022)
- [Adversarial Variational Optimization of Non-Differentiable Simulators](https://mlsanity.com/paper/1707.07113/) (2017)
- [Variational Bayes under Model Misspecification](https://mlsanity.com/paper/1905.10859/) (2019)
- [Neural Empirical Bayes: Source Distribution Estimation and its Applications to Simulation-Based Inference](https://mlsanity.com/paper/2011.05836/) (2020)
- [Variational Bayesian Decision-making for Continuous Utilities](https://mlsanity.com/paper/1902.00792/) (2019)
- [PACm-Bayes: Narrowing the Empirical Risk Gap in the Misspecified Bayesian Regime](https://mlsanity.com/paper/2010.09629/) (2020)
- [Prediction-Centric Uncertainty Quantification via MMD](https://mlsanity.com/paper/2410.11637/) (2024)
- [A General Method for Robust Bayesian Modeling](https://mlsanity.com/paper/1510.05078/) (2015)
- [Holes in Bayesian Statistics](https://mlsanity.com/paper/2002.06467/) (2020)
- [Reproducible Model Selection Using Bagged Posteriors](https://mlsanity.com/paper/2007.14845/) (2020)
- [Simulation-based stacking](https://mlsanity.com/paper/2310.17009/) (2023)
- [A Cheat Sheet for Bayesian Prediction](https://mlsanity.com/paper/2304.12218/) (2023)

## Built on by (most cited first, 3 of 3)

- [Diffusion Models in Simulation-Based Inference: A Tutorial Review](https://mlsanity.com/paper/2512.20685/) (2025)
- [Thinned Mean Field Langevin Dynamics](https://mlsanity.com/paper/2605.28589/) (2026)
- [Soft Specialists: α-R'enyi Ensembles for Uncertainty-Aware LLM Post-Training](https://mlsanity.com/paper/2605.27747/) (2026)

## Abstract (arXiv)

Vanilla variational inference finds an optimal approximation to the Bayesian posterior distribution, but even the exact Bayesian posterior is often not meaningful under model misspecification. We propose predictive variational inference (PVI): a general inference framework that seeks and samples from an optimal posterior density such that the resulting posterior predictive distribution is as close to the true data generating process as possible, while this closeness is measured by multiple scoring rules. By optimizing the objective, the predictive variational inference is generally not the same as, or even attempting to approximate, the Bayesian posterior, even asymptotically. Rather, we interpret it as implicit hierarchical expansion. Further, the learned posterior uncertainty detects heterogeneity of parameters among the population, enabling automatic model diagnosis. This framework applies to both likelihood-exact and likelihood-free models. We demonstrate its application in real data examples.