# Penalty Gradient Normalization for Generative Adversarial Networks

arXiv: https://arxiv.org/abs/2306.13576 · HTML: https://mlsanity.com/paper/2306.13576/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 26 of its 34 references that are in the map.
- First posted on arXiv 23 Jun 2023.

## Builds on (most cited first, 26 of 26)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [Large Scale GAN Training for High Fidelity Natural Image Synthesis](https://mlsanity.com/paper/1809.11096/) (2018)
- [Improved Training of Wasserstein GANs](https://mlsanity.com/paper/1704.00028/) (2017)
- [Conditional Generative Adversarial Nets](https://mlsanity.com/paper/1411.1784/) (2014)
- [Spectral Normalization for Generative Adversarial Networks](https://mlsanity.com/paper/1802.05957/) (2018)
- [Towards Principled Methods for Training Generative Adversarial Networks](https://mlsanity.com/paper/1701.04862/) (2017)
- [Geometric GAN](https://mlsanity.com/paper/1705.02894/) (2017)
- [Generalization and Equilibrium in Generative Adversarial Nets (GANs)](https://mlsanity.com/paper/1703.00573/) (2017)
- [The Shattered Gradients Problem: If resnets are the answer, then what is the question?](https://mlsanity.com/paper/1702.08591/) (2017)
- [Consistency Regularization for Generative Adversarial Networks](https://mlsanity.com/paper/1910.12027/) (2019)
- [Neural Photo Editing with Introspective Adversarial Networks](https://mlsanity.com/paper/1609.07093/) (2016)
- [AutoGAN: Neural Architecture Search for Generative Adversarial Networks](https://mlsanity.com/paper/1908.03835/) (2019)
- [How (not) to Train your Generative Model: Scheduled Sampling, Likelihood, Adversary?](https://mlsanity.com/paper/1511.05101/) (2015)
- [Improved Consistency Regularization for GANs](https://mlsanity.com/paper/2002.04724/) (2020)
- [Adversarial Lipschitz Regularization](https://mlsanity.com/paper/1907.05681/) (2019)
- [A Large-Scale Study on Regularization and Normalization in GANs](https://mlsanity.com/paper/1807.04720/) (2018)
- [Off-Policy Reinforcement Learning for Efficient and Effective GAN Architecture Search](https://mlsanity.com/paper/2007.09180/) (2020)
- [Gradient Normalization for Generative Adversarial Networks](https://mlsanity.com/paper/2109.02235/) (2021)
- [Lipschitz Generative Adversarial Nets](https://mlsanity.com/paper/1902.05687/) (2019)
- [LT-GAN: Self-Supervised GAN with Latent Transformation Detection](https://mlsanity.com/paper/2010.09893/) (2020)
- [Adaptive Weighted Discriminator for Training Generative Adversarial Networks](https://mlsanity.com/paper/2012.03149/) (2020)

## Abstract (arXiv)

In this paper, we propose a novel normalization method called penalty gradient normalization (PGN) to tackle the training instability of Generative Adversarial Networks (GANs) caused by the sharp gradient space. Unlike existing work such as gradient penalty and spectral normalization, the proposed PGN only imposes a penalty gradient norm constraint on the discriminator function, which increases the capacity of the discriminator. Moreover, the proposed penalty gradient normalization can be applied to different GAN architectures with little modification. Extensive experiments on three datasets show that GANs trained with penalty gradient normalization outperform existing methods in terms of both Frechet Inception and Distance and Inception Score.