# A Large-Scale Study on Regularization and Normalization in GANs

arXiv: https://arxiv.org/abs/1807.04720 · HTML: https://mlsanity.com/paper/1807.04720/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Jul 2018.

## Built on by (most cited first, 26 of 26)

- [Projected GANs Converge Faster](https://mlsanity.com/paper/2111.01007/) (2021)
- [Data Collection and Quality Challenges in Deep Learning: A Data-Centric AI Perspective](https://mlsanity.com/paper/2112.06409/) (2021)
- [Maximum-Likelihood Inverse Reinforcement Learning with Finite-Time Guarantees](https://mlsanity.com/paper/2210.01808/) (2022)
- [Sources of Irreproducibility in Machine Learning: A Review](https://mlsanity.com/paper/2204.07610/) (2022)
- [SQIL: Imitation Learning via Reinforcement Learning with Sparse Rewards](https://mlsanity.com/paper/1905.11108/) (2019)
- [Lipschitzness Is All You Need To Tame Off-policy Generative Adversarial Imitation Learning](https://mlsanity.com/paper/2006.16785/) (2020)
- [A survey on GANs for computer vision: Recent research, analysis and taxonomy](https://mlsanity.com/paper/2203.11242/) (2022)
- [SatDM: Synthesizing Realistic Satellite Image with Semantic Layout Conditioning using Diffusion Models](https://mlsanity.com/paper/2309.16812/) (2023)
- [Disentangling Length Bias In Preference Learning Via Response-Conditioned Modeling](https://mlsanity.com/paper/2502.00814/) (2025)
- [Self-Imitation Learning for Robot Tasks with Sparse and Delayed Rewards](https://mlsanity.com/paper/2010.06962/) (2020)
- [Learning Unsupervised Cross-domain Image-to-Image Translation Using a Shared Discriminator](https://mlsanity.com/paper/2102.04699/) (2021)
- [PeaceGAN: A GAN-based Multi-Task Learning Method for SAR Target Image Generation with a Pose Estimator and an Auxiliary Classifier](https://mlsanity.com/paper/2103.15469/) (2021)
- [Iterative Alignment Flows](https://mlsanity.com/paper/2104.07232/) (2021)
- [Cooperative Distribution Alignment via JSD Upper Bound](https://mlsanity.com/paper/2207.02286/) (2022)
- [Towards Practical Non-Adversarial Distribution Matching](https://mlsanity.com/paper/2310.19690/) (2023)
- [Adversarial synthesis based data-augmentation for code-switched spoken language identification](https://mlsanity.com/paper/2205.15747/) (2022)
- [Regularized Soft Actor-Critic for Behavior Transfer Learning](https://mlsanity.com/paper/2209.13224/) (2022)
- [SEAL: SEmantic-Augmented Imitation Learning via Language Model](https://mlsanity.com/paper/2410.02231/) (2024)
- [Understanding Inverse Reinforcement Learning under Overparameterization: Non-Asymptotic Analysis and Global Optimality](https://mlsanity.com/paper/2503.17865/) (2025)
- [FAIL: Flow Matching Adversarial Imitation Learning for Image Generation](https://mlsanity.com/paper/2602.12155/) (2026)
- [Relational Mimic for Visual Adversarial Imitation Learning](https://mlsanity.com/paper/1912.08444/) (2019)
- [Deep Consensus Learning](https://mlsanity.com/paper/2103.08475/) (2021)
- [Learning Spatially-Adaptive Squeeze-Excitation Networks for Image Synthesis and Image Recognition](https://mlsanity.com/paper/2112.14804/) (2021)
- [RCC-GAN: Regularized Compound Conditional GAN for Large-Scale Tabular Data Synthesis](https://mlsanity.com/paper/2205.11693/) (2022)
- [Penalty Gradient Normalization for Generative Adversarial Networks](https://mlsanity.com/paper/2306.13576/) (2023)
- [Expressive Score-Based Priors for Distribution Matching with Geometry-Preserving Regularization](https://mlsanity.com/paper/2506.14607/) (2025)

## Abstract (arXiv)

Generative adversarial networks (GANs) are a class of deep generative models which aim to learn a target distribution in an unsupervised fashion. While they were successfully applied to many problems, training a GAN is a notoriously challenging task and requires a significant number of hyperparameter tuning, neural architecture engineering, and a non-trivial amount of "tricks". The success in many practical applications coupled with the lack of a measure to quantify the failure modes of GANs resulted in a plethora of proposed losses, regularization and normalization schemes, as well as neural architectures. In this work we take a sober view of the current state of GANs from a practical perspective. We discuss and evaluate common pitfalls and reproducibility issues, open-source our code on Github, and provide pre-trained models on TensorFlow Hub.