mlsanityData to 7 Oct 2026

Wasserstein Gradient Flows for Moreau Envelopes of f-Divergences in Reproducing Kernel Hilbert Spaces

arXiv 2402.04613 ↗First posted 7 Feb 2024stat.ML cs.LG math.FAcited by 15

Not on the map. Why?

Abstract (arXiv)

Commonly used f-divergences of measures, e.g., the Kullback-Leibler divergence, are subject to limitations regarding the support of the involved measures. A remedy is regularizing the f-divergence by a squared maximum mean discrepancy (MMD) associated with a characteristic kernel K. We use the kernel mean embedding to show that this regularization can be rewritten as the Moreau envelope of some function on the associated reproducing kernel Hilbert space. Then, we exploit well-known results on Moreau envelopes in Hilbert spaces to analyze the MMD-regularized f-divergences, particularly their gradients. Subsequently, we use our findings to analyze Wasserstein gradient flows of MMD-regularized f-divergences. We provide proof-of-the-concept numerical examples for flows starting from empirical measures. Here, we cover f-divergences with infinite and finite recession constants. Lastly, we extend our results to the tight variational formulation of f-divergences and numerically compare the resulting flows.

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Builds on 22 of its 78 references that are in the map. First posted on arXiv 7 Feb 2024.

Builds on

22 of 78 references.

References · 22

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2016Variational Inference: A Review for Statisticianscited by 1,466
  3. 2016Kernel Mean Embedding of Distributions: A Review and Beyondcited by 360
  4. 2015Optimal Entropy-Transport problems and a new Hellinger-Kantorovich distance between positive measurescited by 138
  5. 2020Generalized Energy Based Modelscited by 73
  6. 2022Mean-Field Langevin Dynamics: Exponential Convergence and Annealingcited by 60
  7. 2019Sinkhorn Divergences for Unbalanced Optimal Transportcited by 52
  8. 2016Kernel Distribution Embeddings: Universal Kernels, Characteristic Kernels and Kernel Metrics on Distributionscited by 39
  9. 2021KALE Flow: A Relaxed KL Gradient Flow for Probabilities with Disjoint Supportcited by 37
  10. 2023Generative Sliced MMD Flows with Riesz Kernelscited by 36
  11. 2020Smoothness and Stability in GANscited by 27
  12. 2020Optimal Bounds between f-Divergences and Integral Probability Metricscited by 27
10 more
  1. 2022Wasserstein Steepest Descent Flows of Discrepancies with Riesz Kernelscited by 16
  2. 2017Strictly proper kernel scores and characteristic kernels on compact spacescited by 14
  3. 2021Controlling Wasserstein Distances by Kernel Norms with Application to Compressive Statistical Learningcited by 13
  4. 2021Invertible Neural Networks versus MCMC for Posterior Reconstruction in Grazing Incidence X-Ray Fluorescencecited by 12
  5. 2021Unbalanced Multi-Marginal Optimal Transportcited by 11
  6. 2022MixFlows: principled variational inference via mixed flowscited by 11
  7. 2021Moreau-Yosida f-divergencescited by 5
  8. 2023Estimation Beyond Data Reweighting: Kernel Method of Momentscited by 5
  9. 2019Curve Based Approximation of Measures on Manifolds by Discrepancy Minimizationcited by 4
  10. 2017Conjugates of integral functionals on continuous functionscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 5; 2025: 5; 2026: 5.

15 papers.

Sort:

  1. 2024Mirror and Preconditioned Gradient Descent in Wasserstein Spacecited by 11
  2. 2025Weighted quantization using MMD: From mean field to mean shift via gradient flowscited by 10
  3. 2024Interaction-Force Transport Gradient Flowscited by 7
  4. 2024Inclusive KL Gradient Flows: Otto-Wasserstein, Fisher-Rao-Gaussian, and Local-Estimator Dynamicscited by 4
  5. 2025DDEQs: Distributional Deep Equilibrium Models through Wasserstein Gradient Flowscited by 4
  6. 2025Ultra-fast feature learning for the training of two-layer neural networks in the two-timescale regimecited by 4
  7. 2024Forward-Euler time-discretization for Wasserstein gradient flows can be wrongcited by 3
  8. 2024Statistical and Geometrical properties of regularized Kernel Kullback-Leibler divergencecited by 3
  9. 2026Sobolev Regularized MMD Gradient Flowcited by 3
  10. 2025Smoothed Distance Kernels for MMDs and Applications in Wasserstein Gradient Flowscited by 2
  11. 2025Kernel Trace Distance: Quantum Statistical Metric between Measures through RKHS Density Operatorscited by 0
  12. 2026HOT-POT: Optimal Transport for Sparse Stereo Matchingcited by 0
3 more
  1. 2026Approximating f-Divergences with Rank Statisticscited by 0
  2. 2026Well-Posed KL-Regularized Control via Wasserstein and Kalman-Wasserstein KL Divergencescited by 0
  3. 2026Graph-Informed Adversarial Modeling: Infimal Subadditivity of Interpolative Divergencescited by 0