# Theoretical Guarantees for Variational Inference with Fixed-Variance Mixture of Gaussians arXiv: https://arxiv.org/abs/2406.04012 ยท HTML: https://mlsanity.com/paper/2406.04012/ ## Key facts - Cited by 11 papers in the map (data through 14 Oct 2026). - Builds on 26 of its 46 references that are in the map. - First posted on arXiv 6 Jun 2024. ## Builds on (most cited first, 26 of 26) - [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016) - [Advances in Variational Inference](https://mlsanity.com/paper/1711.05597/) (2017) - [Handbook of Convergence Theorems for (Stochastic) Gradient Methods](https://mlsanity.com/paper/2301.11235/) (2023) - [Maximum Mean Discrepancy Gradient Flow](https://mlsanity.com/paper/1906.04370/) (2019) - [Sampling as optimization in the space of measures: The Langevin dynamics as a composite optimization problem](https://mlsanity.com/paper/1802.08089/) (2018) - [Variational inference via Wasserstein gradient flows](https://mlsanity.com/paper/2205.15902/) (2022) - [On the geometry of Stein variational gradient descent](https://mlsanity.com/paper/1912.00894/) (2019) - [A Wasserstein-type distance in the space of Gaussian Mixture Models](https://mlsanity.com/paper/1907.05254/) (2019) - [Towards a Theory of Non-Log-Concave Sampling: First-Order Stationarity Guarantees for Langevin Monte Carlo](https://mlsanity.com/paper/2202.05214/) (2022) - [A Non-Asymptotic Analysis for Stein Variational Gradient Descent](https://mlsanity.com/paper/2006.09797/) (2020) - [Kernel Stein Discrepancy Descent](https://mlsanity.com/paper/2105.09994/) (2021) - [The Information Geometry of Mirror Descent](https://mlsanity.com/paper/1310.7780/) (2013) - [Fast and Simple Natural-Gradient Variational Inference with Mixture of Exponential-family Approximations](https://mlsanity.com/paper/1906.02914/) (2019) - [Forward-backward Gaussian variational inference via JKO in the Bures-Wasserstein Space](https://mlsanity.com/paper/2304.05398/) (2023) - [Mirror Descent with Relative Smoothness in Measure Spaces, with application to Sinkhorn and EM](https://mlsanity.com/paper/2206.08873/) (2022) - [Provable convergence guarantees for black-box variational inference](https://mlsanity.com/paper/2306.03638/) (2023) - [Algorithms for mean-field variational inference via polyhedral optimization in the Wasserstein space](https://mlsanity.com/paper/2312.02849/) (2023) - [Mean-field Variational Inference via Wasserstein Gradient Flow](https://mlsanity.com/paper/2207.08074/) (2022) - [Homeomorphic-Invariance of EM: Non-Asymptotic Convergence in KL Divergence for Exponential Families via Mirror Descent](https://mlsanity.com/paper/2011.01170/) (2020) - [Sampling with Mollified Interaction Energy Descent](https://mlsanity.com/paper/2210.13400/) (2022) - [An Explicit Expansion of the Kullback-Leibler Divergence along its Fisher-Rao Gradient Flow](https://mlsanity.com/paper/2302.12229/) (2023) - [A connection between Tempering and Entropic Mirror Descent](https://mlsanity.com/paper/2310.11914/) (2023) - [Independent projections of diffusions: Gradient flows for variational inference and optimal mean field approximations](https://mlsanity.com/paper/2309.13332/) (2023) - [Bridging the Gap Between Variational Inference and Wasserstein Gradient Flows](https://mlsanity.com/paper/2310.20090/) (2023) - [A blob method for inhomogeneous diffusion with applications to multi-agent control and sampling](https://mlsanity.com/paper/2202.12927/) (2022) - [On the use of a local hatR to improve MCMC convergence diagnostic](https://mlsanity.com/paper/2205.06694/) (2022) ## Built on by (most cited first, 11 of 11) - [Gaussian Mixture Flow Matching Models](https://mlsanity.com/paper/2504.05304/) (2025) - [A theoretical perspective on mode collapse in variational inference](https://mlsanity.com/paper/2410.13300/) (2024) - [Efficient, Multimodal, and Derivative-Free Bayesian Inference With Fisher-Rao Gradient Flows](https://mlsanity.com/paper/2406.17263/) (2024) - [Variational Inference with Mixtures of Isotropic Gaussians](https://mlsanity.com/paper/2506.13613/) (2025) - [DDEQs: Distributional Deep Equilibrium Models through Wasserstein Gradient Flows](https://mlsanity.com/paper/2503.01140/) (2025) - [Annealing in variational inference mitigates mode collapse: A theoretical study on Gaussian mixtures](https://mlsanity.com/paper/2602.12923/) (2026) - [A Particle Algorithm for Mean-Field Variational Inference](https://mlsanity.com/paper/2412.20385/) (2024) - [Semi-Implicit Functional Gradient Flow for Efficient Sampling](https://mlsanity.com/paper/2410.17935/) (2024) - [Variational Bayes Portfolio Construction](https://mlsanity.com/paper/2411.06192/) (2024) - [Differentiable Expectation-Maximisation and Applications to Gaussian Mixture Model Optimal Transport](https://mlsanity.com/paper/2509.02109/) (2025) - [Stochastic Gradient Variational Inference with Price's Gradient Estimator from Bures-Wasserstein to Parameter Space](https://mlsanity.com/paper/2602.18718/) (2026) ## Abstract (arXiv) Variational inference (VI) is a popular approach in Bayesian inference, that looks for the best approximation of the posterior distribution within a parametric family, minimizing a loss that is typically the (reverse) Kullback-Leibler (KL) divergence. Despite its empirical success, the theoretical properties of VI have only received attention recently, and mostly when the parametric family is the one of Gaussians. This work aims to contribute to the theoretical study of VI in the non-Gaussian case by investigating the setting of Mixture of Gaussians with fixed covariance and constant weights. In this view, VI over this specific family can be casted as the minimization of a Mollified relative entropy, i.e. the KL between the convolution (with respect to a Gaussian kernel) of an atomic measure supported on Diracs, and the target distribution. The support of the atomic measure corresponds to the localization of the Gaussian components. Hence, solving variational inference becomes equivalent to optimizing the positions of the Diracs (the particles), which can be done through gradient descent and takes the form of an interacting particle system. We study two sources of error of variational inference in this context when optimizing the mollified relative entropy. The first one is an optimization result, that is a descent lemma establishing that the algorithm decreases the objective at each iteration. The second one is an approximation error, that upper bounds the objective between an optimal finite mixture and the target distribution.