# Bayesian Model Selection via Mean-Field Variational Approximation

arXiv: https://arxiv.org/abs/2312.10607 · HTML: https://mlsanity.com/paper/2312.10607/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 40 references that are in the map.
- First posted on arXiv 17 Dec 2023.

## Builds on (most cited first, 17 of 17)

- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [Coordinate Descent Algorithms](https://mlsanity.com/paper/1502.04759/) (2015)
- [On the properties of variational approximations of Gibbs posteriors](https://mlsanity.com/paper/1506.04091/) (2015)
- [Concentration of tempered posteriors and of their variational approximations](https://mlsanity.com/paper/1706.09293/) (2017)
- [Frequentist Consistency of Variational Bayes](https://mlsanity.com/paper/1705.03439/) (2017)
- [Convergence Rates of Variational Posterior Distributions](https://mlsanity.com/paper/1712.02519/) (2017)
- [Singularity, Misspecification, and the Convergence Rate of EM](https://mlsanity.com/paper/1810.00828/) (2018)
- [A Bayesian information criterion for singular models](https://mlsanity.com/paper/1309.0911/) (2013)
- [Variational Bayes under Model Misspecification](https://mlsanity.com/paper/1905.10859/) (2019)
- [Theoretical and Computational Guarantees of Mean Field Variational Inference for Community Detection](https://mlsanity.com/paper/1710.11268/) (2017)
- [On the Computational Complexity of High-Dimensional Bayesian Variable Selection](https://mlsanity.com/paper/1505.07925/) (2015)
- [Consistency of Variational Bayes Inference for Estimation and Model Selection in Mixtures](https://mlsanity.com/paper/1805.05054/) (2018)
- [Particle-based Energetic Variational Inference](https://mlsanity.com/paper/2004.06443/) (2020)
- [Variational Particle Approximations](https://mlsanity.com/paper/1402.5715/) (2014)
- [Singularity structures and impacts on parameter estimation in finite mixtures of distributions](https://mlsanity.com/paper/1609.02655/) (2016)
- [α-Variational Inference with Statistical Guarantees](https://mlsanity.com/paper/1710.03266/) (2017)
- [Fast Convergence of Belief Propagation to Global Optima: Beyond Correlation Decay](https://mlsanity.com/paper/1905.09992/) (2019)

## Built on by (most cited first, 4 of 4)

- [Adaptive sparse variational approximations for Gaussian process regression](https://mlsanity.com/paper/2504.03321/) (2025)
- [Semantic Variational Bayes Based on Semantic Information G Theory for Solving Latent Variables](https://mlsanity.com/paper/2408.13122/) (2024)
- [Rotated Mean-Field Variational Inference and Iterative Gaussianization](https://mlsanity.com/paper/2510.07732/) (2025)
- [Robust Variational Bayes by Min-Max Median Aggregation](https://mlsanity.com/paper/2512.12676/) (2025)

## Abstract (arXiv)

This article considers Bayesian model selection via mean-field (MF) variational approximation. Towards this goal, we study the non-asymptotic properties of MF inference under the Bayesian framework that allows latent variables and model mis-specification. Concretely, we show a Bernstein von-Mises (BvM) theorem for the variational distribution from MF under possible model mis-specification, which implies the distributional convergence of MF variational approximation to a normal distribution centering at the maximal likelihood estimator (within the specified model). Motivated by the BvM theorem, we propose a model selection criterion using the evidence lower bound (ELBO), and demonstrate that the model selected by ELBO tends to asymptotically agree with the one selected by the commonly used Bayesian information criterion (BIC) as sample size tends to infinity. Comparing to BIC, ELBO tends to incur smaller approximation error to the log-marginal likelihood (a.k.a. model evidence) due to a better dimension dependence and full incorporation of the prior information. Moreover, we show the geometric convergence of the coordinate ascent variational inference (CAVI) algorithm under the parametric model framework, which provides a practical guidance on how many iterations one typically needs to run when approximating the ELBO. These findings demonstrate that variational inference is capable of providing a computationally efficient alternative to conventional approaches in tasks beyond obtaining point estimates, which is also empirically demonstrated by our extensive numerical experiments.