mlsanityData to 14 Oct 2026

On the Convergence of Coordinate Ascent Variational Inference

arXiv 2306.01122 ↗First posted 1 Jun 2023stat.ML cs.LG math.STcited by 18

Not on the map. Why?

Abstract (arXiv)

As a computational alternative to Markov chain Monte Carlo approaches, variational inference (VI) is becoming more and more popular for approximating intractable posterior distributions in large-scale Bayesian models due to its comparable efficacy and superior efficiency. Several recent works provide theoretical justifications of VI by proving its statistical optimality for parameter estimation under various settings; meanwhile, formal analysis on the algorithmic convergence aspects of VI is still largely lacking. In this paper, we consider the common coordinate ascent variational inference (CAVI) algorithm for implementing the mean-field (MF) VI towards optimizing a Kullback--Leibler divergence objective functional over the space of all factorized distributions. Focusing on the two-block case, we analyze the convergence of CAVI by leveraging the extensive toolbox from functional analysis and optimization. We provide general conditions for certifying global or local exponential convergence of CAVI. Specifically, a new notion of generalized correlation for characterizing the interaction between the constituting blocks in influencing the VI objective functional is introduced, which according to the theory, quantifies the algorithmic contraction rate of two-block CAVI. As illustrations, we apply the developed theory to a number of examples, and derive explicit problem-dependent upper bounds on the algorithmic contraction rate.

Key facts. Cited by 18 papers in the map (data through 14 Oct 2026). Builds on 13 of its 38 references that are in the map. First posted on arXiv 1 Jun 2023.

Builds on

13 of 38 references.

References · 13

  1. 2016Variational Inference: A Review for Statisticianscited by 1,472
  2. 2013Expectation Propagation for approximate Bayesian inferencecited by 174
  3. 2016R'enyi Divergence Variational Inferencecited by 104
  4. 2017Concentration of tempered posteriors and of their variational approximationscited by 66
  5. 2017Frequentist Consistency of Variational Bayescited by 60
  6. 2017Convergence Rates of Variational Posterior Distributionscited by 45
  7. 2019Variational Bayes for high-dimensional linear regression with sparse priorscited by 44
  8. 2019Randomly initialized EM algorithm for two-component Gaussian mixture achieves near optimality in O(√n) iterationscited by 27
  9. 2017On Statistical Optimality of Variational Bayescited by 21
  10. 2017Theoretical and Computational Guarantees of Mean Field Variational Inference for Community Detectioncited by 19
  11. 2018Wasserstein Variational Inferencecited by 17
  12. 2021Local convexity of the TAP free energy and AMP convergence for Z2-synchronizationcited by 15
1 more
  1. 2018An Instability in Variational Inference for Topic Modelscited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 4; 2024: 4; 2025: 4; 2026: 6.

18 papers.

Sort:

By area:

  1. 2023Algorithms for mean-field variational inference via polyhedral optimization in the Wasserstein spacecited by 20
  2. 2024Convergence of coordinate ascent variational inference for log-concave measures via optimal transportcited by 14
  3. 2023Independent projections of diffusions: Gradient flows for variational inference and optimal mean field approximationscited by 12
  4. 2023Partially factorized variational inference for high-dimensional mixed modelscited by 10
  5. 2024Convergence rate of random scan Coordinate Ascent Variational Inference under log-concavitycited by 9
  6. 2023Bayesian inference for data-efficient, explainable, and safe robotic motion planning: A reviewcited by 1
  7. 2025Combining Bayesian Inference and Reinforcement Learning for Agent Decision Making: A Reviewcited by 1
  8. 2025Exponential Convergence of CAVI for Bayesian PCAcited by 1
  9. 2025Nearly Dimension-Independent Convergence of Mean-Field Black-Box Variational Inferencecited by 1
  10. 2025Rotated Mean-Field Variational Inference and Iterative Gaussianizationcited by 1
  11. 2026Radon--Wasserstein Gradient Flows for Interacting-Particle Sampling in High Dimensionscited by 1
  12. 2026Scalable Mean-Field Variational Inference via Preconditioned Primal-Dual Optimizationcited by 1
6 more
  1. 2026Stability of Sequential and Parallel Coordinate Ascent Variational Inferencecited by 1
  2. 2026Wasserstein Contraction of Coordinate Ascent Variational Inferencecited by 1
  3. 2024Variational Bayesian Bow tie Neural Networks with Shrinkagecited by 0
  4. 2024Variational Bayesian Inference for Tensor Robust Principal Component Analysiscited by 0
  5. 2026Variational Approximated Restricted Maximum Likelihood Estimation for Spatial Datacited by 0
  6. 2026Stability of Finite-Batch Particle Mean-Field Variational Inference Beyond Strong Convexitycited by 0