# On the Effects of Batch and Weight Normalization in Generative Adversarial Networks

arXiv: https://arxiv.org/abs/1704.03971 · HTML: https://mlsanity.com/paper/1704.03971/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Apr 2017.

## Built on by (most cited first, 13 of 13)

- [Large Scale Image Completion via Co-Modulated Generative Adversarial Networks](https://mlsanity.com/paper/2103.10428/) (2021)
- [Dirichlet Energy Constrained Learning for Deep Graph Neural Networks](https://mlsanity.com/paper/2107.02392/) (2021)
- [Gradient Normalization for Generative Adversarial Networks](https://mlsanity.com/paper/2109.02235/) (2021)
- [A Survey on Training Challenges in Generative Adversarial Networks for Biomedical Image Analysis](https://mlsanity.com/paper/2201.07646/) (2022)
- [Attention to Mean-Fields for Particle Cloud Generation](https://mlsanity.com/paper/2305.15254/) (2023)
- [A Realism Metric for Generated LiDAR Point Clouds](https://mlsanity.com/paper/2208.14958/) (2022)
- [Monotone Generative Modeling via a Gromov-Monge Embedding](https://mlsanity.com/paper/2311.01375/) (2023)
- [CHAIN: Enhancing Generalization in Data-Efficient GANs via lipsCHitz continuity constrAIned Normalization](https://mlsanity.com/paper/2404.00521/) (2024)
- [Audio-guided Album Cover Art Generation with Genetic Algorithms](https://mlsanity.com/paper/2207.07162/) (2022)
- [Fully Embedded Time-Series Generative Adversarial Networks](https://mlsanity.com/paper/2308.15730/) (2023)
- [Neural Collage Transfer: Artistic Reconstruction via Material Manipulation](https://mlsanity.com/paper/2311.02202/) (2023)
- [DisCoPatch: Taming Adversarially-driven Batch Statistics for Improved Out-of-Distribution Detection](https://mlsanity.com/paper/2501.08005/) (2025)
- [Flooding Regularization for Stable Training of Generative Adversarial Networks](https://mlsanity.com/paper/2311.00318/) (2023)

## Abstract (arXiv)

Generative adversarial networks (GANs) are highly effective unsupervised learning frameworks that can generate very sharp data, even for data such as images with complex, highly multimodal distributions. However GANs are known to be very hard to train, suffering from problems such as mode collapse and disturbing visual artifacts. Batch normalization (BN) techniques have been introduced to address the training. Though BN accelerates the training in the beginning, our experiments show that the use of BN can be unstable and negatively impact the quality of the trained model. The evaluation of BN and numerous other recent schemes for improving GAN training is hindered by the lack of an effective objective quality measure for GAN models. To address these issues, we first introduce a weight normalization (WN) approach for GAN training that significantly improves the stability, efficiency and the quality of the generated samples. To allow a methodical evaluation, we introduce squared Euclidean reconstruction error on a test set as a new objective measure, to assess training performance in terms of speed, stability, and quality of generated samples. Our experiments with a standard DCGAN architecture on commonly used datasets (CelebA, LSUN bedroom, and CIFAR-10) indicate that training using WN is generally superior to BN for GANs, achieving 10% lower mean squared loss for reconstruction and significantly better qualitative results than BN. We further demonstrate the stability of WN on a 21-layer ResNet trained with the CelebA data set. The code for this paper is available at https://github.com/stormraiser/gan-weightnorm-resnet