# Inclusive KL Gradient Flows: Otto-Wasserstein, Fisher-Rao-Gaussian, and Local-Estimator Dynamics

arXiv: https://arxiv.org/abs/2411.00214 · HTML: https://mlsanity.com/paper/2411.00214/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 57 of its 102 references that are in the map.
- First posted on arXiv 31 Oct 2024.

## Builds on (most cited first, 30 of 57)

- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [Demystifying MMD GANs](https://mlsanity.com/paper/1801.01401/) (2018)
- [Generative Moment Matching Networks](https://mlsanity.com/paper/1502.02761/) (2015)
- [MMD GAN: Towards Deeper Understanding of Moment Matching Network](https://mlsanity.com/paper/1705.08584/) (2017)
- [A Kernelized Stein Discrepancy for Goodness-of-fit Tests and Model Evaluation](https://mlsanity.com/paper/1602.03253/) (2016)
- [Statistical Aspects of Wasserstein Distances](https://mlsanity.com/paper/1806.05500/) (2018)
- [Expectation Propagation for approximate Bayesian inference](https://mlsanity.com/paper/1301.2294/) (2013)
- [Training generative neural networks via Maximum Mean Discrepancy optimization](https://mlsanity.com/paper/1505.03906/) (2015)
- [A Kernel Test of Goodness of Fit](https://mlsanity.com/paper/1602.02964/) (2016)
- [Optimal Entropy-Transport problems and a new Hellinger-Kantorovich distance between positive measures](https://mlsanity.com/paper/1508.07941/) (2015)
- [Maximum Mean Discrepancy Gradient Flow](https://mlsanity.com/paper/1906.04370/) (2019)
- [Measuring Sample Quality with Kernels](https://mlsanity.com/paper/1703.01717/) (2017)
- [The Bayesian Learning Rule](https://mlsanity.com/paper/2107.04562/) (2021)
- [Variational inference via Wasserstein gradient flows](https://mlsanity.com/paper/2205.15902/) (2022)
- [On the geometry of Stein variational gradient descent](https://mlsanity.com/paper/1912.00894/) (2019)
- [Variational Learning is Effective for Large Deep Networks](https://mlsanity.com/paper/2402.17641/) (2024)
- [SVGD as a kernelized Wasserstein gradient flow of the chi-squared divergence](https://mlsanity.com/paper/2006.02509/) (2020)
- [Mean-Field Langevin Dynamics: Exponential Convergence and Annealing](https://mlsanity.com/paper/2202.01009/) (2022)
- [Minimum Stein Discrepancy Estimators](https://mlsanity.com/paper/1906.08283/) (2019)
- [Kernel Stein Discrepancy Descent](https://mlsanity.com/paper/2105.09994/) (2021)
- [Finding Mixed Nash Equilibria of Generative Adversarial Networks](https://mlsanity.com/paper/1811.02002/) (2018)
- [Statistical optimal transport](https://mlsanity.com/paper/2407.18163/) (2024)
- [Markovian Score Climbing: Variational Inference with KL(p||q)](https://mlsanity.com/paper/2003.10374/) (2020)
- [Transformers Learn Nonlinear Features In Context: Nonconvex Mean-field Dynamics on the Attention Landscape](https://mlsanity.com/paper/2402.01258/) (2024)
- [Accelerating Langevin Sampling with Birth-death](https://mlsanity.com/paper/1905.09863/) (2019)
- [Refining Deep Generative Models via Discriminator Gradient Flow](https://mlsanity.com/paper/2012.00780/) (2020)
- [Stein Points](https://mlsanity.com/paper/1803.10161/) (2018)
- [Challenges and Opportunities in High-dimensional Variational Inference](https://mlsanity.com/paper/2103.01085/) (2021)
- [Mirror Descent with Relative Smoothness in Measure Spaces, with application to Sinkhorn and EM](https://mlsanity.com/paper/2206.08873/) (2022)
- [Birth-death dynamics for sampling: Global convergence, approximations and their asymptotics](https://mlsanity.com/paper/2211.00450/) (2022)

## Built on by (most cited first, 4 of 4)

- [Hellinger-Kantorovich Gradient Flows: Global Exponential Decay of Entropy Functionals](https://mlsanity.com/paper/2501.17049/) (2025)
- [A note on the unique properties of the Kullback--Leibler divergence for sampling via gradient flows](https://mlsanity.com/paper/2507.04330/) (2025)
- [Pathway to O(√d) Complexity bound under Wasserstein metric of flow-based models](https://mlsanity.com/paper/2512.06702/) (2025)
- [Rethinking the Flow-Based Gradual Domain Adaptation: A Semi-Dual Optimal Transport Perspective](https://mlsanity.com/paper/2602.01179/) (2026)

## Abstract (arXiv)

Otto's Wasserstein gradient flow of the inclusive (forward) Kullback--Leibler (KL) divergence offers a principled framework for analyzing statistical inference algorithms, yet algorithms targeting the exclusive (reverse) KL divergence are rarely studied with such tools. We establish a unified gradient-flow and PDF framework for inclusive KL inference. We show that maximum mean discrepancy minimization can be viewed as inclusive KL inference with an approximate gradient estimator, and we develop the Fisher--Rao and Wasserstein--Fisher--Rao gradient flows that directly target the inclusive KL divergence. Restricting these flows to the manifold of Gaussian distributions yields explicit gradient-flow ODEs, providing a foundation for Gaussian variational inference. Building on this viewpoint, we further introduce a local-estimator Wasserstein gradient flow whose velocity is obtained by local nonparametric regression, free of density-ratio evaluation or kernel gradients, improving the algorithmic performance over the MMD-based particle method.