mlsanityData to 7 Oct 2026

Bridging the Gap Between Variational Inference and Wasserstein Gradient Flows

arXiv 2310.20090 ↗First posted 31 Oct 2023stat.ML cs.LG stat.COcited by 8

Not on the map. Why?

Abstract (arXiv)

Variational inference is a technique that approximates a target distribution by optimizing within the parameter space of variational families. On the other hand, Wasserstein gradient flows describe optimization within the space of probability measures where they do not necessarily admit a parametric density function. In this paper, we bridge the gap between these two methods. We demonstrate that, under certain conditions, the Bures-Wasserstein gradient flow can be recast as the Euclidean gradient flow where its forward Euler scheme is the standard black-box variational inference algorithm. Specifically, the vector field of the gradient flow is generated via the path-derivative gradient estimator. We also offer an alternative perspective on the path-derivative gradient, framing it as a distillation procedure to the Wasserstein gradient flow. Distillations can be extended to encompass f-divergences and non-Gaussian variational families. This extension yields a new gradient estimator for f-divergences, readily implementable using contemporary machine learning libraries like PyTorch or TensorFlow.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 30 of its 51 references that are in the map. First posted on arXiv 31 Oct 2023.

Builds on

30 of 51 references.

References · 30

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  3. 2016Variational Inference: A Review for Statisticianscited by 1,466
  4. 2016TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systemscited by 660
  5. 2015Importance Weighted Autoencoderscited by 454
  6. 2013Black Box Variational Inferencecited by 317
  7. 2019Monte Carlo Gradient Estimation in Machine Learningcited by 281
  8. 2013Expectation Propagation for approximate Bayesian inferencecited by 172
  9. 2018Sampling as optimization in the space of measures: The Langevin dynamics as a composite optimization problemcited by 123
  10. 2016R'enyi Divergence Variational Inferencecited by 104
  11. 2017Sticking the Landing: Simple, Lower-Variance Gradient Estimators for Variational Inferencecited by 97
  12. 2017Convergence of Langevin MCMC in KL-divergencecited by 86
18 more
  1. 2022Variational inference via Wasserstein gradient flowscited by 84
  2. 2020VarGrad: A Low-Variance Gradient Estimator for Variational Inferencecited by 73
  3. 2020SVGD as a kernelized Wasserstein gradient flow of the chi-squared divergencecited by 64
  4. 2020Markovian Score Climbing: Variational Inference with KL(p||q)cited by 46
  5. 2020Refining Deep Generative Models via Discriminator Gradient Flowcited by 45
  6. 2018Doubly Reparameterized Gradient Estimators for Monte Carlo Objectivescited by 41
  7. 2020Advances in Black-Box VI: Normalizing Flows, Importance Weighting, and Optimizationcited by 37
  8. 2021KALE Flow: A Relaxed KL Gradient Flow for Probabilities with Disjoint Supportcited by 37
  9. 2023Forward-backward Gaussian variational inference via JKO in the Bures-Wasserstein Spacecited by 36
  10. 2021Averaging on the Bures-Wasserstein manifold: dimension-free convergence of gradient descentcited by 32
  11. 2021Variational Refinement for Importance Sampling Using the Forward Kullback-Leibler Divergencecited by 29
  12. 2022Gradients should stay on Path: Better Estimators of the Reverse- and Forward KL Divergence for Normalizing Flowscited by 22
  13. 2016Learning to Draw Samples: With Application to Amortized MLE for Generative Adversarial Learningcited by 21
  14. 2020On the Difficulty of Unbiased Alpha Divergence Minimizationcited by 21
  15. 2019Training Neural Networks for Likelihood/Density Ratio Estimationcited by 16
  16. 2021Generalized Doubly Reparameterized Gradient Estimatorscited by 14
  17. 2023MonoFlow: Rethinking Divergence GANs via the Perspective of Wasserstein Gradient Flowscited by 12
  18. 2022Path-Gradient Estimators for Continuous Normalizing Flowscited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 6; 2025: 0; 2026: 1.

8 papers.

  1. 2023Algorithms for mean-field variational inference via polyhedral optimization in the Wasserstein spacecited by 20
  2. 2024Theoretical Guarantees for Variational Inference with Fixed-Variance Mixture of Gaussianscited by 11
  3. 2024Disentangling impact of capacity, objective, batchsize, estimators, and step-size on flow VIcited by 5
  4. 2024Wasserstein Gradient Boosting: A Framework for Distribution-Valued Supervised Learningcited by 4
  5. 2024Inclusive KL Gradient Flows: Otto-Wasserstein, Fisher-Rao-Gaussian, and Local-Estimator Dynamicscited by 4
  6. 2024A Particle Algorithm for Mean-Field Variational Inferencecited by 3
  7. 2024Stochastic variance-reduced Gaussian variational inference on the Bures-Wasserstein manifoldcited by 2
  8. 2026Stochastic Gradient Variational Inference with Price's Gradient Estimator from Bures-Wasserstein to Parameter Spacecited by 0