mlsanityData to 14 Oct 2026

Inclusive KL Gradient Flows: Otto-Wasserstein, Fisher-Rao-Gaussian, and Local-Estimator Dynamics

arXiv 2411.00214 ↗First posted 31 Oct 2024stat.ML cs.LG math.OCcited by 4

Not on the map. Why?

Abstract (arXiv)

Otto's Wasserstein gradient flow of the inclusive (forward) Kullback--Leibler (KL) divergence offers a principled framework for analyzing statistical inference algorithms, yet algorithms targeting the exclusive (reverse) KL divergence are rarely studied with such tools. We establish a unified gradient-flow and PDF framework for inclusive KL inference. We show that maximum mean discrepancy minimization can be viewed as inclusive KL inference with an approximate gradient estimator, and we develop the Fisher--Rao and Wasserstein--Fisher--Rao gradient flows that directly target the inclusive KL divergence. Restricting these flows to the manifold of Gaussian distributions yields explicit gradient-flow ODEs, providing a foundation for Gaussian variational inference. Building on this viewpoint, we further introduce a local-estimator Wasserstein gradient flow whose velocity is obtained by local nonparametric regression, free of density-ratio evaluation or kernel gradients, improving the algorithmic performance over the MMD-based particle method.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 57 of its 102 references that are in the map. First posted on arXiv 31 Oct 2024.

Builds on

57 of 102 references.

References · 57

  1. 2016Variational Inference: A Review for Statisticianscited by 1,472
  2. 2018Demystifying MMD GANscited by 1,293
  3. 2015Generative Moment Matching Networkscited by 257
  4. 2017MMD GAN: Towards Deeper Understanding of Moment Matching Networkcited by 254
  5. 2016A Kernelized Stein Discrepancy for Goodness-of-fit Tests and Model Evaluationcited by 236
  6. 2018Statistical Aspects of Wasserstein Distancescited by 203
  7. 2013Expectation Propagation for approximate Bayesian inferencecited by 174
  8. 2015Training generative neural networks via Maximum Mean Discrepancy optimizationcited by 151
  9. 2016A Kernel Test of Goodness of Fitcited by 143
  10. 2015Optimal Entropy-Transport problems and a new Hellinger-Kantorovich distance between positive measurescited by 139
  11. 2019Maximum Mean Discrepancy Gradient Flowcited by 127
  12. 2017Measuring Sample Quality with Kernelscited by 119
38 more
  1. 2021The Bayesian Learning Rulecited by 99
  2. 2022Variational inference via Wasserstein gradient flowscited by 85
  3. 2019On the geometry of Stein variational gradient descentcited by 82
  4. 2024Variational Learning is Effective for Large Deep Networkscited by 65
  5. 2020SVGD as a kernelized Wasserstein gradient flow of the chi-squared divergencecited by 64
  6. 2022Mean-Field Langevin Dynamics: Exponential Convergence and Annealingcited by 61
  7. 2019Minimum Stein Discrepancy Estimatorscited by 57
  8. 2021Kernel Stein Discrepancy Descentcited by 57
  9. 2018Finding Mixed Nash Equilibria of Generative Adversarial Networkscited by 49
  10. 2024Statistical optimal transportcited by 47
  11. 2020Markovian Score Climbing: Variational Inference with KL(p||q)cited by 46
  12. 2024Transformers Learn Nonlinear Features In Context: Nonconvex Mean-field Dynamics on the Attention Landscapecited by 46
  13. 2019Accelerating Langevin Sampling with Birth-deathcited by 45
  14. 2020Refining Deep Generative Models via Discriminator Gradient Flowcited by 45
  15. 2018Stein Pointscited by 43
  16. 2021Challenges and Opportunities in High-dimensional Variational Inferencecited by 35
  17. 2022Mirror Descent with Relative Smoothness in Measure Spaces, with application to Sinkhorn and EMcited by 34
  18. 2022Birth-death dynamics for sampling: Global convergence, approximations and their asymptoticscited by 30
  19. 2021Variational Refinement for Importance Sampling Using the Forward Kullback-Leibler Divergencecited by 29
  20. 2020A mean-field analysis of two-player zero-sum gamescited by 28
  21. 2023Learning Gaussian Mixtures Using the Wasserstein-Fisher-Rao Gradient Flowcited by 27
  22. 2024Sampling in Unit Time with Kernel Fisher-Rao Flowcited by 27
  23. 2023Posterior Sampling Based on Gradient Flows of the MMD with Negative Distance Kernelcited by 25
  24. 2015Provable Bayesian Inference via Particle Mirror Descentcited by 23
  25. 2023Unifying GANs and Score-Based Diffusion as Generative Particle Modelscited by 21
  26. 2022Regularized Stein Variational Gradient Flowcited by 19
  27. 2024Deep MMD Gradient Flow without adversarial trainingcited by 19
  28. 2020Estimating Barycenters of Measures in High Dimensionscited by 18
  29. 2023Gradient Flows for Sampling: Mean-Field Models, Gaussian Approximations and Affine Invariancecited by 16
  30. 2023A connection between Tempering and Entropic Mirror Descentcited by 16
  31. 2024(De)-regularized Maximum Mean Discrepancy Gradient Flowcited by 16
  32. 2024Wasserstein Gradient Flows for Moreau Envelopes of f-Divergences in Reproducing Kernel Hilbert Spacescited by 15
  33. 2022An Exponentially Converging Particle Method for the Mixed Nash Equilibrium of Continuous Gamescited by 14
  34. 2023MonoFlow: Rethinking Divergence GANs via the Perspective of Wasserstein Gradient Flowscited by 13
  35. 2024Fisher-Rao Gradient Flow: Geodesic Convexity and Functional Inequalitiescited by 13
  36. 2020Unbalanced Sobolev Descentcited by 11
  37. 2024Stein transport for Bayesian inferencecited by 11
  38. 2018Bayesian posterior approximation via greedy particle optimizationcited by 10

50 most cited of 57 All 57 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 1.

4 papers.

  1. 2025Hellinger-Kantorovich Gradient Flows: Global Exponential Decay of Entropy Functionalscited by 7
  2. 2025A note on the unique properties of the Kullback--Leibler divergence for sampling via gradient flowscited by 4
  3. 2025Pathway to O(√d) Complexity bound under Wasserstein metric of flow-based modelscited by 4
  4. 2026Rethinking the Flow-Based Gradual Domain Adaptation: A Semi-Dual Optimal Transport Perspectivecited by 0