# Mode Collapse of Mean-Field Variational Inference

arXiv: https://arxiv.org/abs/2510.17063 · HTML: https://mlsanity.com/paper/2510.17063/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 39 references that are in the map.
- First posted on arXiv 20 Oct 2025.

## Builds on (most cited first, 18 of 18)

- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [An Introduction to Variational Autoencoders](https://mlsanity.com/paper/1906.02691/) (2019)
- [Frequentist Consistency of Variational Bayes](https://mlsanity.com/paper/1705.03439/) (2017)
- [Beyond ELBOs: A Large-Scale Evaluation of Variational Methods for Sampling](https://mlsanity.com/paper/2406.07423/) (2024)
- [Local Maxima in the Likelihood of Gaussian Mixture Models: Structural Results and Algorithmic Consequences](https://mlsanity.com/paper/1609.00978/) (2016)
- [Learning Mixtures of Gaussians in High Dimensions](https://mlsanity.com/paper/1503.00424/) (2015)
- [On Statistical Optimality of Variational Bayes](https://mlsanity.com/paper/1712.08983/) (2017)
- [Algorithms for mean-field variational inference via polyhedral optimization in the Wasserstein space](https://mlsanity.com/paper/2312.02849/) (2023)
- [Convergence of coordinate ascent variational inference for log-concave measures via optimal transport](https://mlsanity.com/paper/2404.08792/) (2024)
- [A theoretical perspective on mode collapse in variational inference](https://mlsanity.com/paper/2410.13300/) (2024)
- [Independent projections of diffusions: Gradient flows for variational inference and optimal mean field approximations](https://mlsanity.com/paper/2309.13332/) (2023)
- [An Instability in Variational Inference for Topic Models](https://mlsanity.com/paper/1802.00568/) (2018)
- [Convergence rate of random scan Coordinate Ascent Variational Inference under log-concavity](https://mlsanity.com/paper/2406.07292/) (2024)
- [A General Method for Robust Bayesian Modeling](https://mlsanity.com/paper/1510.05078/) (2015)
- [Extending Mean-Field Variational Inference via Entropic Regularization: Theory and Computation](https://mlsanity.com/paper/2404.09113/) (2024)
- [Stability of Mean-Field Variational Inference](https://mlsanity.com/paper/2506.07856/) (2025)
- [When majority rules, minority loses: bias amplification of gradient descent](https://mlsanity.com/paper/2505.13122/) (2025)
- [Rotated Mean-Field Variational Inference and Iterative Gaussianization](https://mlsanity.com/paper/2510.07732/) (2025)

## Abstract (arXiv)

Mean-field variational inference (MFVI) is a widely used method for approximating high-dimensional probability distributions by product measures. It has been empirically observed that MFVI optimizers often suffer from mode collapse. For example, when the target measure π is a mixture π = w P0 + (1 - w) P1, the MFVI optimizer tends to place most of its mass near a single component of the mixture. This work provides the first theoretical explanation of mode collapse in MFVI. Starting from two-component mixtures, we introduce the notion of varepsilon-separateness to quantify the separation of the mixture components. We derive explicit bounds on the mass that any MFVI optimizer assigns to each component, when P0 and P1 are varepsilon-separated for sufficiently small varepsilon. We further extend this analysis to mixture target distributions with multiple components. Our results reveal that the occurrence of mode collapse crucially depends on the relative position of the components. To address this issue, we propose rotational variational inference (RoVI), which augments MFVI with a rotation matrix. The numerical studies support our theoretical findings and illustrate improved mode coverage with RoVI in Bayesian mixture posteriors, including a real-data example.