# Implicit competitive regularization in GANs

arXiv: https://arxiv.org/abs/1910.05852 · HTML: https://mlsanity.com/paper/1910.05852/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Oct 2019.

## Built on by (most cited first, 14 of 14)

- [Particle Cloud Generation with Message Passing Generative Adversarial Networks](https://mlsanity.com/paper/2106.11535/) (2021)
- [Wasserstein GANs Work Because They Fail (to Approximate the Wasserstein Distance)](https://mlsanity.com/paper/2103.01678/) (2021)
- [Free Lunch for Domain Adversarial Training: Environment Label Smoothing](https://mlsanity.com/paper/2302.00194/) (2023)
- [Robust Reinforcement Learning as a Stackelberg Game via Adaptively-Regularized Adversarial Training](https://mlsanity.com/paper/2202.09514/) (2022)
- [Competitive Physics Informed Networks](https://mlsanity.com/paper/2204.11144/) (2022)
- [Discretization Drift in Two-Player Games](https://mlsanity.com/paper/2105.13922/) (2021)
- [Complex Momentum for Optimization in Games](https://mlsanity.com/paper/2102.08431/) (2021)
- [Self-Supervised GANs with Label Augmentation](https://mlsanity.com/paper/2106.08601/) (2021)
- [Generative Minimization Networks: Training GANs Without Competition](https://mlsanity.com/paper/2103.12685/) (2021)
- [Adaptive Consensus Optimization Method for GANs](https://mlsanity.com/paper/2304.10317/) (2023)
- [Implicit regularisation in stochastic gradient descent: from single-objective to two-player games](https://mlsanity.com/paper/2307.05789/) (2023)
- [Generative Adversarial Networks via a Composite Annealing of Noise and Diffusion](https://mlsanity.com/paper/2105.00220/) (2021)
- [Learning to Play General-Sum Games Against Multiple Boundedly Rational Agents](https://mlsanity.com/paper/2106.05492/) (2021)
- [A Gauss-Newton Approach for Min-Max Optimization in Generative Adversarial Networks](https://mlsanity.com/paper/2404.07172/) (2024)

## Abstract (arXiv)

To improve the stability of GAN training we need to understand why they can produce realistic samples. Presently, this is attributed to properties of the divergence obtained under an optimal discriminator. This argument has a fundamental flaw: If we do not impose regularity of the discriminator, it can exploit visually imperceptible errors of the generator to always achieve the maximal generator loss. In practice, gradient penalties are used to regularize the discriminator. However, this needs a metric on the space of images that captures visual similarity. Such a metric is not known, which explains the limited success of gradient penalties in stabilizing GANs. We argue that the performance of GANs is instead due to the implicit competitive regularization (ICR) arising from the simultaneous optimization of generator and discriminator. ICR promotes solutions that look real to the discriminator and thus leverages its inductive biases to generate realistic images. We show that opponent-aware modelling of generator and discriminator, as present in competitive gradient descent (CGD), can significantly strengthen ICR and thus stabilize GAN training without explicit regularization. In our experiments, we use an existing implementation of WGAN-GP and show that by training it with CGD we can improve the inception score (IS) on CIFAR10 for a wide range of scenarios, without any hyperparameter tuning. The highest IS is obtained by combining CGD with the WGAN-loss, without any explicit regularization.