# Wasserstein Gradient Flows for Moreau Envelopes of f-Divergences in Reproducing Kernel Hilbert Spaces

arXiv: https://arxiv.org/abs/2402.04613 · HTML: https://mlsanity.com/paper/2402.04613/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 78 references that are in the map.
- First posted on arXiv 7 Feb 2024.

## Builds on (most cited first, 22 of 22)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [Kernel Mean Embedding of Distributions: A Review and Beyond](https://mlsanity.com/paper/1605.09522/) (2016)
- [Optimal Entropy-Transport problems and a new Hellinger-Kantorovich distance between positive measures](https://mlsanity.com/paper/1508.07941/) (2015)
- [Generalized Energy Based Models](https://mlsanity.com/paper/2003.05033/) (2020)
- [Mean-Field Langevin Dynamics: Exponential Convergence and Annealing](https://mlsanity.com/paper/2202.01009/) (2022)
- [Sinkhorn Divergences for Unbalanced Optimal Transport](https://mlsanity.com/paper/1910.12958/) (2019)
- [Kernel Distribution Embeddings: Universal Kernels, Characteristic Kernels and Kernel Metrics on Distributions](https://mlsanity.com/paper/1604.05251/) (2016)
- [KALE Flow: A Relaxed KL Gradient Flow for Probabilities with Disjoint Support](https://mlsanity.com/paper/2106.08929/) (2021)
- [Generative Sliced MMD Flows with Riesz Kernels](https://mlsanity.com/paper/2305.11463/) (2023)
- [Smoothness and Stability in GANs](https://mlsanity.com/paper/2002.04185/) (2020)
- [Optimal Bounds between f-Divergences and Integral Probability Metrics](https://mlsanity.com/paper/2006.05973/) (2020)
- [Wasserstein Steepest Descent Flows of Discrepancies with Riesz Kernels](https://mlsanity.com/paper/2211.01804/) (2022)
- [Strictly proper kernel scores and characteristic kernels on compact spaces](https://mlsanity.com/paper/1712.05279/) (2017)
- [Controlling Wasserstein Distances by Kernel Norms with Application to Compressive Statistical Learning](https://mlsanity.com/paper/2112.00423/) (2021)
- [Invertible Neural Networks versus MCMC for Posterior Reconstruction in Grazing Incidence X-Ray Fluorescence](https://mlsanity.com/paper/2102.03189/) (2021)
- [Unbalanced Multi-Marginal Optimal Transport](https://mlsanity.com/paper/2103.10854/) (2021)
- [MixFlows: principled variational inference via mixed flows](https://mlsanity.com/paper/2205.07475/) (2022)
- [Moreau-Yosida f-divergences](https://mlsanity.com/paper/2102.13416/) (2021)
- [Estimation Beyond Data Reweighting: Kernel Method of Moments](https://mlsanity.com/paper/2305.10898/) (2023)
- [Curve Based Approximation of Measures on Manifolds by Discrepancy Minimization](https://mlsanity.com/paper/1910.06124/) (2019)
- [Conjugates of integral functionals on continuous functions](https://mlsanity.com/paper/1701.03745/) (2017)

## Built on by (most cited first, 15 of 15)

- [Mirror and Preconditioned Gradient Descent in Wasserstein Space](https://mlsanity.com/paper/2406.08938/) (2024)
- [Weighted quantization using MMD: From mean field to mean shift via gradient flows](https://mlsanity.com/paper/2502.10600/) (2025)
- [Interaction-Force Transport Gradient Flows](https://mlsanity.com/paper/2405.17075/) (2024)
- [Sobolev Regularized MMD Gradient Flow](https://mlsanity.com/paper/2605.11884/) (2026)
- [Ultra-fast feature learning for the training of two-layer neural networks in the two-timescale regime](https://mlsanity.com/paper/2504.18208/) (2025)
- [Inclusive KL Gradient Flows: Otto-Wasserstein, Fisher-Rao-Gaussian, and Local-Estimator Dynamics](https://mlsanity.com/paper/2411.00214/) (2024)
- [DDEQs: Distributional Deep Equilibrium Models through Wasserstein Gradient Flows](https://mlsanity.com/paper/2503.01140/) (2025)
- [Forward-Euler time-discretization for Wasserstein gradient flows can be wrong](https://mlsanity.com/paper/2406.08209/) (2024)
- [Statistical and Geometrical properties of regularized Kernel Kullback-Leibler divergence](https://mlsanity.com/paper/2408.16543/) (2024)
- [Smoothed Distance Kernels for MMDs and Applications in Wasserstein Gradient Flows](https://mlsanity.com/paper/2504.07820/) (2025)
- [Kernel Trace Distance: Quantum Statistical Metric between Measures through RKHS Density Operators](https://mlsanity.com/paper/2507.06055/) (2025)
- [HOT-POT: Optimal Transport for Sparse Stereo Matching](https://mlsanity.com/paper/2601.12423/) (2026)
- [Approximating f-Divergences with Rank Statistics](https://mlsanity.com/paper/2601.22784/) (2026)
- [Well-Posed KL-Regularized Control via Wasserstein and Kalman-Wasserstein KL Divergences](https://mlsanity.com/paper/2602.02250/) (2026)
- [Graph-Informed Adversarial Modeling: Infimal Subadditivity of Interpolative Divergences](https://mlsanity.com/paper/2603.20025/) (2026)

## Abstract (arXiv)

Commonly used f-divergences of measures, e.g., the Kullback-Leibler divergence, are subject to limitations regarding the support of the involved measures. A remedy is regularizing the f-divergence by a squared maximum mean discrepancy (MMD) associated with a characteristic kernel K. We use the kernel mean embedding to show that this regularization can be rewritten as the Moreau envelope of some function on the associated reproducing kernel Hilbert space. Then, we exploit well-known results on Moreau envelopes in Hilbert spaces to analyze the MMD-regularized f-divergences, particularly their gradients. Subsequently, we use our findings to analyze Wasserstein gradient flows of MMD-regularized f-divergences. We provide proof-of-the-concept numerical examples for flows starting from empirical measures. Here, we cover f-divergences with infinite and finite recession constants. Lastly, we extend our results to the tight variational formulation of f-divergences and numerically compare the resulting flows.