# Frequentist Consistency of Variational Bayes

arXiv: https://arxiv.org/abs/1705.03439 · HTML: https://mlsanity.com/paper/1705.03439/

## Key facts

- Cited by 60 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 May 2017.

## Built on by (most cited first, 30 of 60)

- [Deep End-to-end Causal Inference](https://mlsanity.com/paper/2202.02195/) (2022)
- [Variational inference via Wasserstein gradient flows](https://mlsanity.com/paper/2205.15902/) (2022)
- [Simplifying Deep Temporal Difference Learning](https://mlsanity.com/paper/2407.04811/) (2024)
- [Desiderata for Representation Learning: A Causal Perspective](https://mlsanity.com/paper/2109.03795/) (2021)
- [Disentangling Identifiable Features from Noisy Data with Structured Nonlinear ICA](https://mlsanity.com/paper/2106.09620/) (2021)
- [EdgeBench: Unveiling Scaling Laws of Learning from Real-World Environments](https://mlsanity.com/paper/2607.05155/) (2026)
- [An Adaptive Kernel Approach to Federated Learning of Heterogeneous Causal Effects](https://mlsanity.com/paper/2301.00346/) (2023)
- [On the Convergence of Coordinate Ascent Variational Inference](https://mlsanity.com/paper/2306.01122/) (2023)
- [On the Robustness to Misspecification of α-Posteriors and Their Variational Approximations](https://mlsanity.com/paper/2104.08324/) (2021)
- [Local convexity of the TAP free energy and AMP convergence for Z2-synchronization](https://mlsanity.com/paper/2106.11428/) (2021)
- [Variational Inference for Uncertainty Quantification: an Analysis of Trade-offs](https://mlsanity.com/paper/2403.13748/) (2024)
- [Variational Inference in high-dimensional linear regression](https://mlsanity.com/paper/2104.12232/) (2021)
- [Generalization Gap in Amortized Inference](https://mlsanity.com/paper/2205.11640/) (2022)
- [Variational Inference in Location-Scale Families: Exact Recovery of the Mean and Correlation Matrix](https://mlsanity.com/paper/2410.11067/) (2024)
- [The computational asymptotics of Gaussian variational inference and the Laplace approximation](https://mlsanity.com/paper/2104.05886/) (2021)
- [Approximation-Aware Bayesian Optimization](https://mlsanity.com/paper/2406.04308/) (2024)
- [On the inability of Gaussian process regression to optimally learn compositional functions](https://mlsanity.com/paper/2205.07764/) (2022)
- [Statistical and Computational Trade-offs in Variational Inference: A Case Study in Inferential Model Selection](https://mlsanity.com/paper/2207.11208/) (2022)
- [Optimizing Pessimism in Dynamic Treatment Regimes: A Bayesian Learning Approach](https://mlsanity.com/paper/2210.14420/) (2022)
- [Machine Learning and the Future of Bayesian Computation](https://mlsanity.com/paper/2304.11251/) (2023)
- [Spike-and-slab shrinkage priors for structurally sparse Bayesian neural networks](https://mlsanity.com/paper/2308.09104/) (2023)
- [A Mean Field Approach to Empirical Bayes Estimation in High-dimensional Linear Regression](https://mlsanity.com/paper/2309.16843/) (2023)
- [Bayesian Model Selection via Mean-Field Variational Approximation](https://mlsanity.com/paper/2312.10607/) (2023)
- [Spike and slab Bayesian sparse principal component analysis](https://mlsanity.com/paper/2102.00305/) (2021)
- [A fast asynchronous MCMC sampler for sparse Bayesian inference](https://mlsanity.com/paper/2108.06446/) (2021)
- [Nonparametric Automatic Differentiation Variational Inference with Spline Approximation](https://mlsanity.com/paper/2403.06302/) (2024)
- [A Primer on Variational Inference for Physics-Informed Deep Generative Modelling](https://mlsanity.com/paper/2409.06560/) (2024)
- [Stability of Mean-Field Variational Inference](https://mlsanity.com/paper/2506.07856/) (2025)
- [Generalized Guarantees for Variational Inference in the Presence of Even and Elliptical Symmetry](https://mlsanity.com/paper/2511.01064/) (2025)
- [Structured Optimal Variational Inference for Dynamic Latent Space Models](https://mlsanity.com/paper/2209.15117/) (2022)

## Abstract (arXiv)

A key challenge for modern Bayesian statistics is how to perform scalable inference of posterior distributions. To address this challenge, variational Bayes (VB) methods have emerged as a popular alternative to the classical Markov chain Monte Carlo (MCMC) methods. VB methods tend to be faster while achieving comparable predictive performance. However, there are few theoretical results around VB. In this paper, we establish frequentist consistency and asymptotic normality of VB methods. Specifically, we connect VB methods to point estimates based on variational approximations, called frequentist variational approximations, and we use the connection to prove a variational Bernstein-von Mises theorem. The theorem leverages the theoretical characterizations of frequentist variational approximations to understand asymptotic properties of VB. In summary, we prove that (1) the VB posterior converges to the Kullback-Leibler (KL) minimizer of a normal distribution, centered at the truth and (2) the corresponding variational expectation of the parameter is consistent and asymptotically normal. As applications of the theorem, we derive asymptotic properties of VB posteriors in Bayesian mixture models, Bayesian generalized linear mixed models, and Bayesian stochastic block models. We conduct a simulation study to illustrate these theoretical results.