mlsanityData to 14 Oct 2026

Generalization Error of GAN from the Discriminator's Perspective

arXiv 2107.03633 ↗First posted 8 Jul 2021cs.LG stat.MLcited by 8

Not on the map. Why?

Abstract (arXiv)

The generative adversarial network (GAN) is a well-known model for learning high-dimensional distributions, but the mechanism for its generalization ability is not understood. In particular, GAN is vulnerable to the memorization phenomenon, the eventual convergence to the empirical distribution. We consider a simplified GAN model with the generator replaced by a density, and analyze how the discriminator contributes to generalization. We show that with early stopping, the generalization error measured by Wasserstein metric escapes from the curse of dimensionality, despite that in the long term, memorization is inevitable. In addition, we present a hardness of learning result for WGAN.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 39 of its 67 references that are in the map. First posted on arXiv 8 Jul 2021.

Builds on

39 of 67 references.

References · 39

  1. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
  2. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  3. 2016Layer Normalizationcited by 5,669
  4. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,527
  5. 2013Auto-Encoding Variational Bayescited by 4,053
  6. 2017Improved Training of Wasserstein GANscited by 2,235
  7. 2015Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networkscited by 2,054
  8. 2018Spectral Normalization for Generative Adversarial Networkscited by 1,636
  9. 2018Which Training Methods for GANs do actually Converge?cited by 679
  10. 2017Wasserstein GANcited by 639
  11. 2016f-GAN: Training Generative Neural Samplers using Variational Divergence Minimizationcited by 506
  12. 2019On Gradient Descent Ascent for Nonconvex-Concave Minimax Problemscited by 270
27 more
  1. 2017Generalization and Equilibrium in Generative Adversarial Nets (GANs)cited by 166
  2. 2019Maximum Mean Discrepancy Gradient Flowcited by 127
  3. 2016Mode Regularized Generative Adversarial Networkscited by 123
  4. 2016Energy-based Generative Adversarial Networkcited by 119
  5. 2020Towards a Mathematical Understanding of Neural Network-Based Machine Learning: what we know and what we don'tcited by 95
  6. 2017Gradient descent GAN optimization is locally stablecited by 67
  7. 2017On the regularization of Wasserstein GANscited by 65
  8. 2020Generative Adversarial Networks (GANs Survey): Challenges, Solutions, and Future Directionscited by 65
  9. 2017GP-GAN: Towards Realistic High-Resolution Image Blendingcited by 64
  10. 2018Solving Linear Inverse Problems Using GAN Priors: An Algorithm with Provable Guaranteescited by 58
  11. 2019Machine Learning from a Continuous Viewpointcited by 50
  12. 2017On the Discrimination-Generalization Tradeoff in GANscited by 48
  13. 2020On the Convergence of Gradient Descent Training for Two-layer ReLU-networks in the Mean Field Regimecited by 39
  14. 2020On the Banach spaces associated with multi-layer ReLU networks: Function representation, approximation theory and gradient descent dynamicscited by 39
  15. 2020Towards GAN Benchmarks Which Require Generalizationcited by 31
  16. 2018Approximability of Discriminators Implies Diversity in GANscited by 28
  17. 2020Kolmogorov Width Decay and Poor Approximators in Machine Learning: Shallow Neural Networks, Random Feature Models and Neural Tangent Kernelscited by 19
  18. 2017On the Effectiveness of Least Squares Generative Adversarial Networkscited by 18
  19. 2017CosmoGAN: creating high-fidelity weak lensing convergence maps using Generative Adversarial Networkscited by 17
  20. 2019SGD Learns One-Layer Networks in WGANscited by 16
  21. 2019A Priori Estimates of the Population Risk for Residual Networkscited by 14
  22. 2021Understanding Overparameterization in Generative Adversarial Networkscited by 14
  23. 2020Empirical Analysis of Overfitting and Mode Drop in GAN Trainingcited by 11
  24. 2019Learning Distributions Generated by One-Layer ReLU Networkscited by 10
  25. 2020Generalization and Memorization: The Bias Potential Modelcited by 9
  26. 2019The Generalization Error of the Minimum-norm Solutions for Over-parameterized Neural Networkscited by 6
  27. 2017SGAN: An Alternative Training of Generative Adversarial Networkscited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 3; 2023: 3; 2024: 1; 2025: 0; 2026: 1.

8 papers.

  1. 2023On the Stability of Iterative Retraining of Generative Models on their own Datacited by 100
  2. 2023On the Generalization Properties of Diffusion Modelscited by 68
  3. 2023PUGAN: Physical Model-Guided Underwater Image Enhancement Using GAN with Dual-Discriminatorscited by 49
  4. 2022A Mathematical Framework for Learning Probability Distributionscited by 7
  5. 2022Reducing Training Sample Memorization in GANs by Training with Memorization Rejectioncited by 4
  6. 2024Early Stopping Criteria for Training Generative Adversarial Networks in Biomedical Imagingcited by 1
  7. 2022Adaptive adversarial training method for improving multi-scale GAN based on generalization bound theorycited by 0
  8. 2026Generalization and Memorization in Rectified Flowcited by 0