# Many Paths to Equilibrium: GANs Do Not Need to Decrease a Divergence At Every Step

arXiv: https://arxiv.org/abs/1710.08446 · HTML: https://mlsanity.com/paper/1710.08446/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Oct 2017.

## Built on by (most cited first, 30 of 45)

- [Diffusion-GAN: Training GANs with Diffusion](https://mlsanity.com/paper/2206.02262/) (2022)
- [ViTGAN: Training GANs with Vision Transformers](https://mlsanity.com/paper/2107.04589/) (2021)
- [Can Push-forward Generative Models Fit Multimodal Distributions?](https://mlsanity.com/paper/2206.14476/) (2022)
- [RoMA: Robust Model Adaptation for Offline Model-based Optimization](https://mlsanity.com/paper/2110.14188/) (2021)
- [Why neural networks find simple solutions: the many regularizers of geometric complexity](https://mlsanity.com/paper/2209.13083/) (2022)
- [Sample-Efficient Imitation Learning via Generative Adversarial Nets](https://mlsanity.com/paper/1809.02064/) (2018)
- [GATSBI: Generative Adversarial Training for Simulation-Based Inference](https://mlsanity.com/paper/2203.06481/) (2022)
- [Wasserstein GANs Work Because They Fail (to Approximate the Wasserstein Distance)](https://mlsanity.com/paper/2103.01678/) (2021)
- [Improving GAN Equilibrium by Raising Spatial Awareness](https://mlsanity.com/paper/2112.00718/) (2021)
- [VILD: Variational Imitation Learning with Diverse-quality Demonstrations](https://mlsanity.com/paper/1909.06769/) (2019)
- [A Behavior Regularized Implicit Policy for Offline Reinforcement Learning](https://mlsanity.com/paper/2202.09673/) (2022)
- [DiffRatio: Training One-Step Diffusion Models Without Teacher Supervision](https://mlsanity.com/paper/2502.08005/) (2025)
- [The Geometric Occam's Razor Implicit in Deep Learning](https://mlsanity.com/paper/2111.15090/) (2021)
- [Learning to Address Intra-segment Misclassification in Retinal Imaging](https://mlsanity.com/paper/2104.12138/) (2021)
- [Game of GANs: Game-Theoretical Models for Generative Adversarial Networks](https://mlsanity.com/paper/2106.06976/) (2021)
- [Look ATME: The Discriminator Mean Entropy Needs Attention](https://mlsanity.com/paper/2304.09024/) (2023)
- [Anomaly Detection in Additive Manufacturing Processes using Supervised Classification with Imbalanced Sensor Data based on Generative Adversarial Network](https://mlsanity.com/paper/2210.17274/) (2022)
- [GLeaD: Improving GANs with A Generator-Leading Task](https://mlsanity.com/paper/2212.03752/) (2022)
- [Stabilizing GANs' Training with Brownian Motion Controller](https://mlsanity.com/paper/2306.10468/) (2023)
- [ChildGAN: Large Scale Synthetic Child Facial Data Using Domain Adaptation in StyleGAN](https://mlsanity.com/paper/2307.13746/) (2023)
- [Evolving GAN Formulations for Higher Quality Image Synthesis](https://mlsanity.com/paper/2102.08578/) (2021)
- [Learning Polynomial Transformations](https://mlsanity.com/paper/2204.04209/) (2022)
- [A Wasserstein perspective of Vanilla GANs](https://mlsanity.com/paper/2403.15312/) (2024)
- [Generalization of GANs and overparameterized models under Lipschitz continuity](https://mlsanity.com/paper/2104.02388/) (2021)
- [Generative Adversarial Network for Probabilistic Forecast of Random Dynamical System](https://mlsanity.com/paper/2111.03126/) (2021)
- [Data Interpolants -- That's What Discriminators in Higher-order Gradient-regularized GANs Are](https://mlsanity.com/paper/2306.00785/) (2023)
- [OT-Net: A Reusable Neural Optimal Transport Solver](https://mlsanity.com/paper/2306.08233/) (2023)
- [Modeling stochastic eye tracking data: A comparison of quantum generative adversarial networks and Markov models](https://mlsanity.com/paper/2408.00673/) (2024)
- [Modeling Eye Gaze Velocity Trajectories using GANs with Spectral Loss for Enhanced Fidelity](https://mlsanity.com/paper/2412.04184/) (2024)
- [SFBD-OMNI: Bridge models for lossy measurement restoration with limited clean samples](https://mlsanity.com/paper/2512.17051/) (2025)

## Abstract (arXiv)

Generative adversarial networks (GANs) are a family of generative models that do not minimize a single training criterion. Unlike other generative models, the data distribution is learned via a game between a generator (the generative model) and a discriminator (a teacher providing training signal) that each minimize their own cost. GANs are designed to reach a Nash equilibrium at which each player cannot reduce their cost without changing the other players' parameters. One useful approach for the theory of GANs is to show that a divergence between the training distribution and the model distribution obtains its minimum value at equilibrium. Several recent research directions have been motivated by the idea that this divergence is the primary guide for the learning process and that every step of learning should decrease the divergence. We show that this view is overly restrictive. During GAN training, the discriminator provides learning signal in situations where the gradients of the divergences between distributions would not be useful. We provide empirical counterexamples to the view of GAN training as divergence minimization. Specifically, we demonstrate that GANs are able to learn distributions in situations where the divergence minimization point of view predicts they would fail. We also show that gradient penalties motivated from the divergence minimization perspective are equally helpful when applied in other contexts in which the divergence minimization perspective does not predict they would be helpful. This contributes to a growing body of evidence that GAN training may be more usefully viewed as approaching Nash equilibria via trajectories that do not necessarily minimize a specific divergence at each step.