# Functional Space Analysis of Local GAN Convergence

arXiv: https://arxiv.org/abs/2102.04448 · HTML: https://mlsanity.com/paper/2102.04448/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 35 references that are in the map.
- First posted on arXiv 8 Feb 2021.

## Builds on (most cited first, 18 of 18)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Improved Training of Wasserstein GANs](https://mlsanity.com/paper/1704.00028/) (2017)
- [Spectral Normalization for Generative Adversarial Networks](https://mlsanity.com/paper/1802.05957/) (2018)
- [Training Generative Adversarial Networks with Limited Data](https://mlsanity.com/paper/2006.06676/) (2020)
- [Least Squares Generative Adversarial Networks](https://mlsanity.com/paper/1611.04076/) (2016)
- [Which Training Methods for GANs do actually Converge?](https://mlsanity.com/paper/1801.04406/) (2018)
- [Consistency Regularization for Generative Adversarial Networks](https://mlsanity.com/paper/1910.12027/) (2019)
- [Stabilizing Training of Generative Adversarial Networks through Regularization](https://mlsanity.com/paper/1705.09367/) (2017)
- [The Mechanics of n-Player Differentiable Games](https://mlsanity.com/paper/1802.05642/) (2018)
- [The Numerics of GANs](https://mlsanity.com/paper/1705.10461/) (2017)
- [Image Augmentations for GAN Training](https://mlsanity.com/paper/2006.02595/) (2020)
- [Interaction Matters: A Note on Non-asymptotic Local Convergence of Generative Adversarial Networks](https://mlsanity.com/paper/1802.06132/) (2018)
- [Gradient descent GAN optimization is locally stable](https://mlsanity.com/paper/1706.04156/) (2017)
- [Instance Selection for GANs](https://mlsanity.com/paper/2007.15255/) (2020)
- [Training Generative Adversarial Networks by Solving Ordinary Differential Equations](https://mlsanity.com/paper/2010.15040/) (2020)

## Built on by (most cited first, 4 of 4)

- [Instability and Local Minima in GAN Training with Kernel Discriminators](https://mlsanity.com/paper/2208.09938/) (2022)
- [The Impact of Geometric Complexity on Neural Collapse in Transfer Learning](https://mlsanity.com/paper/2405.15706/) (2024)
- [Monotone Generative Modeling via a Gromov-Monge Embedding](https://mlsanity.com/paper/2311.01375/) (2023)
- [Control of Overfitting with Physics](https://mlsanity.com/paper/2412.10716/) (2024)

## Abstract (arXiv)

Recent work demonstrated the benefits of studying continuous-time dynamics governing the GAN training. However, this dynamics is analyzed in the model parameter space, which results in finite-dimensional dynamical systems. We propose a novel perspective where we study the local dynamics of adversarial training in the general functional space and show how it can be represented as a system of partial differential equations. Thus, the convergence properties can be inferred from the eigenvalues of the resulting differential operator. We show that these eigenvalues can be efficiently estimated from the target dataset before training. Our perspective reveals several insights on the practical tricks commonly used to stabilize GANs, such as gradient penalty, data augmentation, and advanced integration schemes. As an immediate practical benefit, we demonstrate how one can a priori select an optimal data augmentation strategy for a particular generation task.