# Instability and Local Minima in GAN Training with Kernel Discriminators

arXiv: https://arxiv.org/abs/2208.09938 · HTML: https://mlsanity.com/paper/2208.09938/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 40 references that are in the map.
- First posted on arXiv 21 Aug 2022.

## Builds on (most cited first, 21 of 21)

- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Neural Tangent Kernel: Convergence and Generalization in Neural Networks](https://mlsanity.com/paper/1806.07572/) (2018)
- [Improved Training of Wasserstein GANs](https://mlsanity.com/paper/1704.00028/) (2017)
- [Demystifying MMD GANs](https://mlsanity.com/paper/1801.01401/) (2018)
- [Which Training Methods for GANs do actually Converge?](https://mlsanity.com/paper/1801.04406/) (2018)
- [Towards Principled Methods for Training Generative Adversarial Networks](https://mlsanity.com/paper/1701.04862/) (2017)
- [NIPS 2016 Tutorial: Generative Adversarial Networks](https://mlsanity.com/paper/1701.00160/) (2016)
- [MMD GAN: Towards Deeper Understanding of Moment Matching Network](https://mlsanity.com/paper/1705.08584/) (2017)
- [On the Inductive Bias of Neural Tangent Kernels](https://mlsanity.com/paper/1905.12173/) (2019)
- [Pros and Cons of GAN Evaluation Measures: New Developments](https://mlsanity.com/paper/2103.09396/) (2021)
- [The Numerics of GANs](https://mlsanity.com/paper/1705.10461/) (2017)
- [Gradient descent GAN optimization is locally stable](https://mlsanity.com/paper/1706.04156/) (2017)
- [On GANs and GMMs](https://mlsanity.com/paper/1805.12462/) (2018)
- [On the Convergence of Gradient Descent in GANs: MMD GAN As a Gradient Flow](https://mlsanity.com/paper/2011.02402/) (2020)
- [A Neural Tangent Kernel Perspective of GANs](https://mlsanity.com/paper/2106.05566/) (2021)
- [Coulomb GANs: Provably Optimal Nash Equilibria via Potential Fields](https://mlsanity.com/paper/1708.08819/) (2017)
- [On Convergence and Stability of GANs](https://mlsanity.com/paper/1705.07215/) (2017)
- [On Catastrophic Forgetting and Mode Collapse in Generative Adversarial Networks](https://mlsanity.com/paper/1807.04015/) (2018)
- [Understanding Overparameterization in Generative Adversarial Networks](https://mlsanity.com/paper/2104.05605/) (2021)
- [Understanding and Stabilizing GANs' Training Dynamics with Control Theory](https://mlsanity.com/paper/1909.13188/) (2019)
- [Functional Space Analysis of Local GAN Convergence](https://mlsanity.com/paper/2102.04448/) (2021)

## Built on by (most cited first, 7 of 7)

- [A Survey on Diffusion Models for Recommender Systems](https://mlsanity.com/paper/2409.05033/) (2024)
- [Disentangling representations of retinal images with generative models](https://mlsanity.com/paper/2402.19186/) (2024)
- [Bridging User Dynamics: Transforming Sequential Recommendations with Schr"odinger Bridge and Diffusion Models](https://mlsanity.com/paper/2409.10522/) (2024)
- [Eval4DiRec: A Unified and Systematic Evaluation Framework for Diffusion-based Recommender Systems](https://mlsanity.com/paper/2609.34404/) (2026)
- [Local Convergence of Gradient Descent-Ascent for Training Generative Adversarial Networks](https://mlsanity.com/paper/2305.08277/) (2023)
- [Generalizable 7T T1-map Synthesis from 1.5T and 3T T1 MRI with an Efficient Transformer Model](https://mlsanity.com/paper/2507.08655/) (2025)
- [An Efficient 3D Latent Diffusion Model for T1-contrast Enhanced MRI Generation](https://mlsanity.com/paper/2509.24194/) (2025)

## Abstract (arXiv)

Generative Adversarial Networks (GANs) are a widely-used tool for generative modeling of complex data. Despite their empirical success, the training of GANs is not fully understood due to the min-max optimization of the generator and discriminator. This paper analyzes these joint dynamics when the true samples, as well as the generated samples, are discrete, finite sets, and the discriminator is kernel-based. A simple yet expressive framework for analyzing training called the textitIsolated Points Model is introduced. In the proposed model, the distance between true samples greatly exceeds the kernel width, so each generated point is influenced by at most one true point. Our model enables precise characterization of the conditions for convergence, both to good and bad minima. In particular, the analysis explains two common failure modes: (i) an approximate mode collapse and (ii) divergence. Numerical simulations are provided that predictably replicate these behaviors.