# Gradient penalty from a maximum margin perspective

arXiv: https://arxiv.org/abs/1910.06922 · HTML: https://mlsanity.com/paper/1910.06922/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Oct 2019.

## Built on by (most cited first, 2 of 2)

- [3D Shape Generation with Grid-based Implicit Functions](https://mlsanity.com/paper/2107.10607/) (2021)
- [Complex Momentum for Optimization in Games](https://mlsanity.com/paper/2102.08431/) (2021)

## Abstract (arXiv)

A popular heuristic for improved performance in Generative adversarial networks (GANs) is to use some form of gradient penalty on the discriminator. This gradient penalty was originally motivated by a Wasserstein distance formulation. However, the use of gradient penalty in other GAN formulations is not well motivated. We present a unifying framework of expected margin maximization and show that a wide range of gradient-penalized GANs (e.g., Wasserstein, Standard, Least-Squares, and Hinge GANs) can be derived from this framework. Our results imply that employing gradient penalties induces a large-margin classifier (thus, a large-margin discriminator in GANs). We describe how expected margin maximization helps reduce vanishing gradients at fake (generated) samples, a known problem in GANs. From this framework, we derive a new L^∞ gradient norm penalty with Hinge loss which generally produces equally good (or better) generated output in GANs than L2-norm penalties (based on the Fr'echet Inception Distance).