# Variational Bayes under Model Misspecification

arXiv: https://arxiv.org/abs/1905.10859 · HTML: https://mlsanity.com/paper/1905.10859/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 May 2019.

## Built on by (most cited first, 22 of 22)

- [Deep End-to-end Causal Inference](https://mlsanity.com/paper/2202.02195/) (2022)
- [Variational Bayesian Last Layers](https://mlsanity.com/paper/2404.11599/) (2024)
- [On the Convergence of Black-Box Variational Inference](https://mlsanity.com/paper/2305.15349/) (2023)
- [Generative Artificial Intelligence for Navigating Synthesizable Chemical Space](https://mlsanity.com/paper/2410.03494/) (2024)
- [On the Robustness to Misspecification of α-Posteriors and Their Variational Approximations](https://mlsanity.com/paper/2104.08324/) (2021)
- [Variational Inference in high-dimensional linear regression](https://mlsanity.com/paper/2104.12232/) (2021)
- [On the inability of Gaussian process regression to optimally learn compositional functions](https://mlsanity.com/paper/2205.07764/) (2022)
- [Statistical and Computational Trade-offs in Variational Inference: A Case Study in Inferential Model Selection](https://mlsanity.com/paper/2207.11208/) (2022)
- [A Comprehensive Guide to Simulation-based Inference in Computational Biology](https://mlsanity.com/paper/2409.19675/) (2024)
- [A Mean Field Approach to Empirical Bayes Estimation in High-dimensional Linear Regression](https://mlsanity.com/paper/2309.16843/) (2023)
- [Bayesian Model Selection via Mean-Field Variational Approximation](https://mlsanity.com/paper/2312.10607/) (2023)
- [Predictive variational inference: Learn the predictively optimal posterior distribution](https://mlsanity.com/paper/2410.14843/) (2024)
- [Scalable Semi-Modular Inference with Variational Meta-Posteriors](https://mlsanity.com/paper/2204.00296/) (2022)
- [Evaluating uncertainties in electrochemical impedance spectra of solid oxide fuel cells](https://mlsanity.com/paper/2101.08049/) (2021)
- [Risk-averse Contextual Multi-armed Bandit Problem with Linear Payoffs](https://mlsanity.com/paper/2206.12463/) (2022)
- [Adaptive time series forecasting with markovian variance switching](https://mlsanity.com/paper/2402.14684/) (2024)
- [Nearly Dimension-Independent Convergence of Mean-Field Black-Box Variational Inference](https://mlsanity.com/paper/2505.21721/) (2025)
- [Even More Guarantees for Variational Inference in the Presence of Symmetries](https://mlsanity.com/paper/2604.21407/) (2026)
- [Identifying Information from Observations with Uncertainty and Novelty](https://mlsanity.com/paper/2501.09331/) (2025)
- [Variational bagging: a robust approach for Bayesian uncertainty quantification](https://mlsanity.com/paper/2511.20594/) (2025)
- [Robust Variational Bayes by Min-Max Median Aggregation](https://mlsanity.com/paper/2512.12676/) (2025)
- [Questioning the Coverage-Length Metric in Conformal Prediction: When Shorter Intervals Are Not Better](https://mlsanity.com/paper/2601.21455/) (2026)

## Abstract (arXiv)

Variational Bayes (VB) is a scalable alternative to Markov chain Monte Carlo (MCMC) for Bayesian posterior inference. Though popular, VB comes with few theoretical guarantees, most of which focus on well-specified models. However, models are rarely well-specified in practice. In this work, we study VB under model misspecification. We prove the VB posterior is asymptotically normal and centers at the value that minimizes the Kullback-Leibler (KL) divergence to the true data-generating distribution. Moreover, the VB posterior mean centers at the same value and is also asymptotically normal. These results generalize the variational Bernstein--von Mises theorem [29] to misspecified models. As a consequence of these results, we find that the model misspecification error dominates the variational approximation error in VB posterior predictive distributions. It explains the widely observed phenomenon that VB achieves comparable predictive accuracy with MCMC even though VB uses an approximating family. As illustrations, we study VB under three forms of model misspecification, ranging from model over-/under-dispersion to latent dimensionality misspecification. We conduct two simulation studies that demonstrate the theoretical results.