mlsanityData to 7 Oct 2026

Gradient Normalization for Generative Adversarial Networks

arXiv 2109.02235 ↗First posted 6 Sep 2021cs.LGcited by 25

Not on the map. Why?

Abstract (arXiv)

In this paper, we propose a novel normalization method called gradient normalization (GN) to tackle the training instability of Generative Adversarial Networks (GANs) caused by the sharp gradient space. Unlike existing work such as gradient penalty and spectral normalization, the proposed GN only imposes a hard 1-Lipschitz constraint on the discriminator function, which increases the capacity of the discriminator. Moreover, the proposed gradient normalization can be applied to different GAN architectures with little modification. Extensive experiments on four datasets show that GANs trained with gradient normalization outperform existing methods in terms of both Frechet Inception Distance and Inception Score.

Key facts. Cited by 25 papers in the map (data through 7 Oct 2026). Builds on 24 of its 36 references that are in the map. First posted on arXiv 6 Sep 2021.

Builds on

24 of 36 references.

References · 24

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  3. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,590
  4. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,967
  5. 2016Improved Techniques for Training GANscited by 2,938
  6. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  7. 2017Improved Training of Wasserstein GANscited by 2,223
  8. 2016Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Networkcited by 1,898
  9. 2018Spectral Normalization for Generative Adversarial Networkscited by 1,618
  10. 2015LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loopcited by 1,350
  11. 2015Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)cited by 1,061
  12. 2018Which Training Methods for GANs do actually Converge?cited by 672
12 more
  1. 2017Towards Principled Methods for Training Generative Adversarial Networkscited by 460
  2. 2018Lipschitz regularity of deep neural networks: analysis and efficient estimationcited by 413
  3. 2019MirrorGAN: Learning Text-to-image Generation by Redescriptioncited by 176
  4. 2019Consistency Regularization for Generative Adversarial Networkscited by 134
  5. 2016Neural Photo Editing with Introspective Adversarial Networkscited by 109
  6. 2019AutoGAN: Neural Architecture Search for Generative Adversarial Networkscited by 94
  7. 2018Improving the Improved Training of Wasserstein GANs: A Consistency Term and Its Dual Effectcited by 49
  8. 2019Spectral Regularization for Combating Mode Collapse in GANscited by 31
  9. 2019Adversarial Lipschitz Regularizationcited by 29
  10. 2020Off-Policy Reinforcement Learning for Efficient and Effective GAN Architecture Searchcited by 25
  11. 2017Wasserstein Divergence for GANscited by 22
  12. 2017On the Effects of Batch and Weight Normalization in Generative Adversarial Networkscited by 13

Built on by

Cited by year

Citing papers per year: 2021: 3; 2022: 7; 2023: 11; 2024: 1; 2025: 3.

25 papers.

Sort:

By area:

  1. 2023Decoupled Rationalization with Asymmetric Learning Rates: A Flexible Lipschitz Restraintcited by 14
  2. 2023Robust low-rank training via approximate orthonormal constraintscited by 12
  3. 2022SCAM! Transferring humans between images with Semantic Cross Attention Modulationcited by 10
  4. 2023Balanced Training for Sparse GANscited by 8
  5. 2021Generative Convolution Layer for Image Generationcited by 5
  6. 2025Scalable GANs with Transformerscited by 5
  7. 2021A Novel Generator with Auxiliary Branch for Improving GAN Performancecited by 4
  8. 2022BMD-GAN: Bone mineral density estimation using x-ray image decomposition into projections of bone-segmented quantitative computed tomography using hierarchical learningcited by 3
  9. 2022GNPassGAN: Improved Generative Adversarial Networks For Trawling Offline Password Guessingcited by 3
  10. 2023Bone mineral density estimation from a plain X-ray image by learning decomposition into projections of bone-segmented computed tomographycited by 3
  11. 2025Continuous-Time Analysis of Heavy Ball Momentum in Min-Max Gamescited by 3
  12. 2022Robust Offline Reinforcement Learning with Gradient Penalty and Constraint Relaxationcited by 2
13 more
  1. 2023Leveraging Contaminated Datasets to Learn Clean-Data Distribution with Purified Generative Adversarial Networkscited by 2
  2. 2023Structural Adversarial Objectives for Self-Supervised Representation Learningcited by 2
  3. 2021GraN-GAN: Piecewise Gradient Normalization for Generative Adversarial Networkscited by 1
  4. 2022On Deep Learning in Password Guessing, a Surveycited by 1
  5. 2024A Comprehensive Survey on Synthetic Infrared Image synthesiscited by 1
  6. 2025Diversifying Long Prompt Image Generation through Structured Prompt Embedding Space Samplingcited by 1
  7. 2022Effective Shortcut Technique for GANcited by 0
  8. 2022Improving GANs with a Feature Cycling Generatorcited by 0
  9. 2023Penalty Gradient Normalization for Generative Adversarial Networkscited by 0
  10. 2023Stability Analysis Framework for Particle-based Distance GANs with Wasserstein Gradient Flowcited by 0
  11. 2023EqGAN: Feature Equalization Fusion for Few-shot Image Generationcited by 0
  12. 2023Accelerated Neural Network Training with Rooted Logistic Objectivescited by 0
  13. 2023Neural Collage Transfer: Artistic Reconstruction via Material Manipulationcited by 0