mlsanityData to 7 Oct 2026

Forward-backward Gaussian variational inference via JKO in the Bures-Wasserstein Space

arXiv 2304.05398 ↗First posted 10 Apr 2023math.ST cs.LG math.OCcited by 36

Not on the map. Why?

Abstract (arXiv)

Variational inference (VI) seeks to approximate a target distribution π by an element of a tractable family of distributions. Of key interest in statistics and machine learning is Gaussian VI, which approximates π by minimizing the Kullback-Leibler (KL) divergence to π over the space of Gaussians. In this work, we develop the (Stochastic) Forward-Backward Gaussian Variational Inference (FB-GVI) algorithm to solve Gaussian VI. Our approach exploits the composite structure of the KL divergence, which can be written as the sum of a smooth term (the potential) and a non-smooth term (the entropy) over the Bures-Wasserstein (BW) space of Gaussians endowed with the Wasserstein distance. For our proposed algorithm, we obtain state-of-the-art convergence guarantees when π is log-smooth and log-concave, as well as the first convergence guarantees to first-order stationary solutions when π is only log-smooth.

Key facts. Cited by 36 papers in the map (data through 7 Oct 2026). Builds on 48 of its 88 references that are in the map. First posted on arXiv 10 Apr 2023.

Builds on

48 of 88 references.

References · 48

  1. 2015Variational Inference with Normalizing Flowscited by 2,161
  2. 2016Variational Inference: A Review for Statisticianscited by 1,466
  3. 2016Optimization Methods for Large-Scale Machine Learningcited by 1,140
  4. 2016Improving Variational Inference with Inverse Autoregressive Flowcited by 636
  5. 2016Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithmcited by 498
  6. 2013Black Box Variational Inferencecited by 317
  7. 2014Theoretical guarantees for approximate sampling from smooth and log-concave densitiescited by 206
  8. 2017Stein Variational Gradient Descent as Gradient Flowcited by 149
  9. 2018Analysis of Langevin Monte Carlo via convex optimizationcited by 126
  10. 2018Sampling as optimization in the space of measures: The Langevin dynamics as a composite optimization problemcited by 123
  11. 2017Noisy Natural Gradient as Variational Inferencecited by 94
  12. 2019A Unified Theory of SGD: Variance Reduction, Sampling, Quantization and Coordinate Descentcited by 84
36 more
  1. 2022Variational inference via Wasserstein gradient flowscited by 84
  2. 2019On the geometry of Stein variational gradient descentcited by 82
  3. 2022Improved analysis for a proximal algorithm for samplingcited by 72
  4. 2017Concentration of tempered posteriors and of their variational approximationscited by 66
  5. 2020SVGD as a kernelized Wasserstein gradient flow of the chi-squared divergencecited by 64
  6. 2022Towards a Theory of Non-Log-Concave Sampling: First-Order Stationarity Guarantees for Langevin Monte Carlocited by 63
  7. 2020A Non-Asymptotic Analysis for Stein Variational Gradient Descentcited by 61
  8. 2020Efficient constrained sampling via the mirror-Langevin algorithmcited by 55
  9. 2021Optimizing Functionals on the Space of Probabilities with Input Convex Neural Networkscited by 53
  10. 2019Fast and Simple Natural-Gradient Variational Inference with Mixture of Exponential-family Approximationscited by 51
  11. 2018Scaling limit of the Stein variational gradient descent: the mean field regimecited by 35
  12. 2021On Riemannian Optimization over Positive Definite Matrices with the Bures-Wasserstein Geometrycited by 32
  13. 2021Averaging on the Bures-Wasserstein manifold: dimension-free convergence of gradient descentcited by 32
  14. 2020Handling the Positive-Definite Constraint in the Bayesian Learning Rulecited by 31
  15. 2019Provable Smoothness Guarantees for Black-Box Variational Inferencecited by 29
  16. 2020Fast Matrix Square Roots with Applications to Gaussian Processes and Bayesian Optimizationcited by 26
  17. 2021Projected Wasserstein gradient descent for high-dimensional Bayesian inferencecited by 24
  18. 2022A Finite-Particle Convergence Rate for Stein Variational Gradient Descentcited by 22
  19. 2021A Convergence Theory for SVGD in the Population Limit under Talagrand's Inequality T1cited by 21
  20. 2022Mean-field Variational Inference via Wasserstein Gradient Flowcited by 19
  21. 2022Regularized Stein Variational Gradient Flowcited by 18
  22. 2019A Generalization Bound for Online Variational Inferencecited by 17
  23. 2022Fast Differentiable Matrix Square Root and Inverse Square Rootcited by 16
  24. 2015The Proximal Robbins-Monro Methodcited by 14
  25. 2020Variational Inference with Continuously-Indexed Normalizing Flowscited by 14
  26. 2015Towards stability and optimality in stochastic gradient descentcited by 11
  27. 2018Gaussian variational approximation for high-dimensional state space modelscited by 10
  28. 2020A Manifold Proximal Linear Method for Sparse Spectral Clustering with Application to Single-Cell RNA Sequencing Data Analysiscited by 7
  29. 2022A Constraint Dissolving Approach for Nonsmooth Optimization over the Stiefel Manifoldcited by 7
  30. 2020Manifold Proximal Point Algorithms for Dual Principal Component Pursuit and Orthogonal Dictionary Learningcited by 6
  31. 2022Bures-Wasserstein Barycenters and Low-Rank Matrix Recoverycited by 5
  32. 2021Weak topology and Opial property in Wasserstein spaces, with applications to Gradient Flows and Proximal Point Algorithms of geodesically convex functionalscited by 4
  33. 2020Riemannian Stochastic Variance-Reduced Cubic Regularized Newton Method for Submanifold Optimizationcited by 3
  34. 2022Nonconvex Matrix Factorization is Geodesically Convex: Global Landscape Analysis for Fixed-rank Matrix Optimization From a Riemannian Perspectivecited by 3
  35. 2017A constant step Forward-Backward algorithm involving random maximal monotone operatorscited by 1
  36. 2021A Riemannian smoothing steepest descent method for non-Lipschitz optimization on submanifoldscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 8; 2024: 10; 2025: 11; 2026: 7.

36 papers.

Sort:

By area:

  1. 2023Convergence of flow-based generative models via proximal gradient descent in Wasserstein spacecited by 34
  2. 2023Provable convergence guarantees for black-box variational inferencecited by 29
  3. 2023On the Convergence of Black-Box Variational Inferencecited by 22
  4. 2024Batch and match: black-box variational inference with a score-based divergencecited by 22
  5. 2023Algorithms for mean-field variational inference via polyhedral optimization in the Wasserstein spacecited by 20
  6. 2025Solving Inverse Problems via Diffusion-Based Priors: An Approximation-Free Ensemble Sampling Approachcited by 18
  7. 2023Towards Understanding the Dynamics of Gaussian-Stein Variational Gradient Descentcited by 13
  8. 2023Momentum Particle Maximum Likelihoodcited by 12
  9. 2024Theoretical Guarantees for Variational Inference with Fixed-Variance Mixture of Gaussianscited by 11
  10. 2024Variational Inference in Location-Scale Families: Exact Recovery of the Mean and Correlation Matrixcited by 9
  11. 2023Bridging the Gap Between Variational Inference and Wasserstein Gradient Flowscited by 8
  12. 2024Proximal Interacting Particle Langevin Algorithmscited by 8
24 more
  1. 2024Sampling from the Mean-Field Stationary Distributioncited by 7
  2. 2025Learning of Population Dynamics: Inverse Optimization Meets JKO Schemecited by 6
  3. 2025Variational Inference with Mixtures of Isotropic Gaussianscited by 6
  4. 2024Non-geodesically-convex optimization in the Wasserstein spacecited by 5
  5. 2023A Mean Field Approach to Empirical Bayes Estimation in High-dimensional Linear Regressioncited by 4
  6. 2024Provably Scalable Black-Box Variational Inference with Structured Variational Familiescited by 4
  7. 2025Training Latent Diffusion Models with Interacting Particle Algorithmscited by 4
  8. 2024A Particle Algorithm for Mean-Field Variational Inferencecited by 3
  9. 2025Optimization Guarantees for Square-Root Natural-Gradient Variational Inferencecited by 3
  10. 2024Stochastic variance-reduced Gaussian variational inference on the Bures-Wasserstein manifoldcited by 2
  11. 2025Variational Online Mirror Descent for Robust Learning in Schr"odinger Bridgecited by 2
  12. 2025Dense associative memory for Gaussian distributionscited by 2
  13. 2025Nearly Dimension-Independent Convergence of Mean-Field Black-Box Variational Inferencecited by 1
  14. 2025Busemann Functions in the Wasserstein Space: Existence, Closed-Forms, and Applications to Slicingcited by 1
  15. 2026DC-LA: Difference-of-Convex Langevin Algorithmcited by 1
  16. 2026Large-scale Score-based Variational Posterior Inference for Bayesian Deep Neural Networkscited by 1
  17. 2026Convergence of projected stochastic natural gradient variational inference for various step size and sample or batch size schedulescited by 1
  18. 2024You Only Accept Samples Once: Fast, Self-Correcting Stochastic Variational Inferencecited by 0
  19. 2025A Smoothing Newton Method for Rank-one Matrix Recoverycited by 0
  20. 2025Gaussian mixture layers for neural networkscited by 0
  21. 2026Bures-Wasserstein Importance-Weighted Evidence Lower Bound: Exposition and Applicationscited by 0
  22. 2026Stochastic Gradient Variational Inference with Price's Gradient Estimator from Bures-Wasserstein to Parameter Spacecited by 0
  23. 2026Large-scale empirical tuning and comparison of default optimizers for variational inferencecited by 0
  24. 2026ITSPACE: Monotone Gaussian Optimal Transport Updatescited by 0