# A Systematic Survey of Regularization and Normalization in GANs

arXiv: https://arxiv.org/abs/2008.08930 · HTML: https://mlsanity.com/paper/2008.08930/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Aug 2020.

## Built on by (most cited first, 27 of 27)

- [Is Artificial Intelligence Generated Image Detection a Solved Problem?](https://mlsanity.com/paper/2505.12335/) (2025)
- [A Survey of Mix-based Data Augmentation: Taxonomy, Methods, Applications, and Explainability](https://mlsanity.com/paper/2212.10888/) (2022)
- [A Comprehensive Survey on Data-Efficient GANs in Image Generation](https://mlsanity.com/paper/2204.08329/) (2022)
- [Domain Re-Modulation for Few-Shot Generative Domain Adaptation](https://mlsanity.com/paper/2302.02550/) (2023)
- [FakeCLR: Exploring Contrastive Learning for Solving Latent Discontinuity in Data-Efficient GANs](https://mlsanity.com/paper/2207.08630/) (2022)
- [Exploring The Effect of High-frequency Components in GANs Training](https://mlsanity.com/paper/2103.11093/) (2021)
- [Image Synthesis under Limited Data: A Survey and Taxonomy](https://mlsanity.com/paper/2307.16879/) (2023)
- [Pedestrian Trajectory Prediction Based on Social Interactions Learning With Random Weights](https://mlsanity.com/paper/2501.07711/) (2025)
- [Efficient Backdoor Attacks for Deep Neural Networks in Real-world Scenarios](https://mlsanity.com/paper/2306.08386/) (2023)
- [Proxy-Tuning: Tailoring Multimodal Autoregressive Models for Subject-Driven Image Generation](https://mlsanity.com/paper/2503.10125/) (2025)
- [A Proxy Attack-Free Strategy for Practically Improving the Poisoning Efficiency in Backdoor Attacks](https://mlsanity.com/paper/2306.08313/) (2023)
- [Addressing GAN Training Instabilities via Tunable Classification Losses](https://mlsanity.com/paper/2310.18291/) (2023)
- [Peer is Your Pillar: A Data-unbalanced Conditional GANs for Few-shot Image Generation](https://mlsanity.com/paper/2311.08217/) (2023)
- [Dual-view X-ray Detection: Can AI Detect Prohibited Items from Dual-view X-ray Images like Humans?](https://mlsanity.com/paper/2411.18082/) (2024)
- [One-shot Generative Domain Adaptation in 3D GANs](https://mlsanity.com/paper/2410.08824/) (2024)
- [OmniForce: On Human-Centered, Large Model Empowered and Cloud-Edge Collaborative AutoML System](https://mlsanity.com/paper/2303.00501/) (2023)
- [Efficient Trigger Word Insertion](https://mlsanity.com/paper/2311.13957/) (2023)
- [Large Language Models are Good Attackers: Efficient and Stealthy Textual Backdoor Attacks](https://mlsanity.com/paper/2408.11587/) (2024)
- [Fault Detection in New Wind Turbines with Limited Data by Generative Transfer Learning](https://mlsanity.com/paper/2504.17709/) (2025)
- [Adversarial Defense in Cybersecurity: A Systematic Review of GANs for Threat Detection and Mitigation](https://mlsanity.com/paper/2509.20411/) (2025)
- [Improving Few-shot Image Generation by Structural Discrimination and Textural Modulation](https://mlsanity.com/paper/2308.16110/) (2023)
- [Flooding Regularization for Stable Training of Generative Adversarial Networks](https://mlsanity.com/paper/2311.00318/) (2023)
- [Maximum Discrepancy Generative Regularization and Non-Negative Matrix Factorization for Single Channel Source Separation](https://mlsanity.com/paper/2404.15296/) (2024)
- [Revising Second Order Terms in Deep Animation Video Coding](https://mlsanity.com/paper/2510.23561/) (2025)
- [How Noise Benefits AI-generated Image Detection](https://mlsanity.com/paper/2511.16136/) (2025)
- [Enhancing Diversity and Feasibility: Joint Population Synthesis from Multi-source Data Using Generative Models](https://mlsanity.com/paper/2602.15270/) (2026)
- [BGA: A noise-immune neural distillation framework for malicious signature extraction in high-entropy encrypted flows](https://mlsanity.com/paper/2608.14126/) (2026)

## Abstract (arXiv)

Generative Adversarial Networks (GANs) have been widely applied in different scenarios thanks to the development of deep neural networks. The original GAN was proposed based on the non-parametric assumption of the infinite capacity of networks. However, it is still unknown whether GANs can fit the target distribution without any prior information. Due to the overconfident assumption, many issues remain unaddressed in GANs' training, such as non-convergence, mode collapses, gradient vanishing. Regularization and normalization are common methods of introducing prior information to stabilize training and improve discrimination. Although a handful number of regularization and normalization methods have been proposed for GANs, to the best of our knowledge, there exists no comprehensive survey that primarily focuses on objectives and development of these methods, apart from some in-comprehensive and limited scope studies. In this work, we conduct a comprehensive survey on the regularization and normalization techniques from different perspectives of GANs training. First, we systematically describe different perspectives of GANs training and thus obtain the different objectives of regularization and normalization. Based on these objectives, we propose a new taxonomy. Furthermore, we compare the performance of the mainstream methods on different datasets and investigate the applications of regularization and normalization techniques that have been frequently employed in state-of-the-art GANs. Finally, we highlight potential future directions of research in this domain. Code and studies related to the regularization and normalization of GANs in this work is summarized on https://github.com/iceli1007/GANs-Regularization-Review.