# Local Convergence of Gradient Descent-Ascent for Training Generative Adversarial Networks

arXiv: https://arxiv.org/abs/2305.08277 · HTML: https://mlsanity.com/paper/2305.08277/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 18 references that are in the map.
- First posted on arXiv 14 May 2023.

## Builds on (most cited first, 13 of 13)

- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [Neural Tangent Kernel: Convergence and Generalization in Neural Networks](https://mlsanity.com/paper/1806.07572/) (2018)
- [Spectral Normalization for Generative Adversarial Networks](https://mlsanity.com/paper/1802.05957/) (2018)
- [Which Training Methods for GANs do actually Converge?](https://mlsanity.com/paper/1801.04406/) (2018)
- [f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization](https://mlsanity.com/paper/1606.00709/) (2016)
- [Towards Principled Methods for Training Generative Adversarial Networks](https://mlsanity.com/paper/1701.04862/) (2017)
- [MMD GAN: Towards Deeper Understanding of Moment Matching Network](https://mlsanity.com/paper/1705.08584/) (2017)
- [Gradient descent GAN optimization is locally stable](https://mlsanity.com/paper/1706.04156/) (2017)
- [On the Convergence of Gradient Descent in GANs: MMD GAN As a Gradient Flow](https://mlsanity.com/paper/2011.02402/) (2020)
- [Coulomb GANs: Provably Optimal Nash Equilibria via Potential Fields](https://mlsanity.com/paper/1708.08819/) (2017)
- [Instability and Local Minima in GAN Training with Kernel Discriminators](https://mlsanity.com/paper/2208.09938/) (2022)

## Abstract (arXiv)

Generative Adversarial Networks (GANs) are a popular formulation to train generative models for complex high dimensional data. The standard method for training GANs involves a gradient descent-ascent (GDA) procedure on a minimax optimization problem. This procedure is hard to analyze in general due to the nonlinear nature of the dynamics. We study the local dynamics of GDA for training a GAN with a kernel-based discriminator. This convergence analysis is based on a linearization of a non-linear dynamical system that describes the GDA iterations, under an textitisolated points model assumption from [Becker et al. 2022]. Our analysis brings out the effect of the learning rates, regularization, and the bandwidth of the kernel discriminator, on the local convergence rate of GDA. Importantly, we show phase transitions that indicate when the system converges, oscillates, or diverges. We also provide numerical simulations that verify our claims.