# On the Convergence of Gradient Descent in GANs: MMD GAN As a Gradient Flow

arXiv: https://arxiv.org/abs/2011.02402 · HTML: https://mlsanity.com/paper/2011.02402/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Nov 2020.

## Built on by (most cited first, 21 of 21)

- [Optimizing Functionals on the Space of Probabilities with Input Convex Neural Networks](https://mlsanity.com/paper/2106.00774/) (2021)
- [A Neural Tangent Kernel Perspective of GANs](https://mlsanity.com/paper/2106.05566/) (2021)
- [Gaussian Interpolation Flows](https://mlsanity.com/paper/2311.11475/) (2023)
- [MonoFlow: Rethinking Divergence GANs via the Perspective of Wasserstein Gradient Flows](https://mlsanity.com/paper/2302.01075/) (2023)
- [Dynamic Flows on Curved Space Generated by Labeled Data](https://mlsanity.com/paper/2302.00061/) (2023)
- [Instability and Local Minima in GAN Training with Kernel Discriminators](https://mlsanity.com/paper/2208.09938/) (2022)
- [Sobolev Regularized MMD Gradient Flow](https://mlsanity.com/paper/2605.11884/) (2026)
- [GANs as Gradient Flows that Converge](https://mlsanity.com/paper/2205.02910/) (2022)
- [Generative Adversarial Networks and Adversarial Autoencoders: Tutorial and Survey](https://mlsanity.com/paper/2111.13282/) (2021)
- [A Theory of Unsupervised Speech Recognition](https://mlsanity.com/paper/2306.07926/) (2023)
- [Interpolation for Robust Learning: Data Augmentation on Wasserstein Geodesics](https://mlsanity.com/paper/2302.02092/) (2023)
- [Generative Modeling with Flow-Guided Density Ratio Learning](https://mlsanity.com/paper/2303.03714/) (2023)
- [Data Interpolants -- That's What Discriminators in Higher-order Gradient-regularized GANs Are](https://mlsanity.com/paper/2306.00785/) (2023)
- [Non-parametric online market regime detection and regime clustering for multidimensional and path-dependent data structures](https://mlsanity.com/paper/2306.15835/) (2023)
- [Adversarial Subspace Generation for Outlier Detection in High-Dimensional Data](https://mlsanity.com/paper/2504.07522/) (2025)
- [On the Optimization Landscape of Maximum Mean Discrepancy](https://mlsanity.com/paper/2110.13452/) (2021)
- [Proximal Mean Field Learning in Shallow Neural Networks](https://mlsanity.com/paper/2210.13879/) (2022)
- [Effective Dynamics of Generative Adversarial Networks](https://mlsanity.com/paper/2212.04580/) (2022)
- [Local Convergence of Gradient Descent-Ascent for Training Generative Adversarial Networks](https://mlsanity.com/paper/2305.08277/) (2023)
- [Stability Analysis Framework for Particle-based Distance GANs with Wasserstein Gradient Flow](https://mlsanity.com/paper/2307.01879/) (2023)
- [One-Step Generative Modeling via Unbalanced Optimal Transport](https://mlsanity.com/paper/2609.32708/) (2026)

## Abstract (arXiv)

We consider the maximum mean discrepancy (MMD) GAN problem and propose a parametric kernelized gradient flow that mimics the min-max game in gradient regularized MMD GAN. We show that this flow provides a descent direction minimizing the MMD on a statistical manifold of probability distributions. We then derive an explicit condition which ensures that gradient descent on the parameter space of the generator in gradient regularized MMD GAN is globally convergent to the target distribution. Under this condition, we give non asymptotic convergence results of gradient descent in MMD GAN. Another contribution of this paper is the introduction of a dynamic formulation of a regularization of MMD and demonstrating that the parametric kernelized descent for MMD is the gradient flow of this functional with respect to the new Riemannian structure. Our obtained theoretical result allows ones to treat gradient flows for quite general functionals and thus has potential applications to other types of variational inferences on a statistical manifold beyond GANs. Finally, numerical experiments suggest that our parametric kernelized gradient flow stabilizes GAN training and guarantees convergence.