mlsanityData to 7 Oct 2026

The Numerics of GANs

arXiv 1705.10461 ↗First posted 30 May 2017cs.LGcited by 104

Not on the map. Why?

Abstract (arXiv)

In this paper, we analyze the numerics of common algorithms for training Generative Adversarial Networks (GANs). Using the formalism of smooth two-player games we analyze the associated gradient vector field of GAN training objectives. Our findings suggest that the convergence of current algorithms suffers due to two factors: i) presence of eigenvalues of the Jacobian of the gradient vector field with zero real-part, and ii) eigenvalues with big imaginary part. Using these findings, we design a new algorithm that overcomes some of these limitations and has better convergence properties. Experimentally, we demonstrate its superiority on training common GAN architectures and show convergence on GAN architectures that are known to be notoriously hard to train.

Key facts. Cited by 104 papers in the map (data through 7 Oct 2026). First posted on arXiv 30 May 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 33; 2022: 22; 2023: 22; 2024: 12; 2025: 5; 2026: 10.

104 papers.

Sort:

By area:

  1. 2022Diffusion-GAN: Training GANs with Diffusioncited by 203
  2. 2021Regularizing Generative Adversarial Networks under Limited Datacited by 105
  3. 2021Aligning Latent and Image Spaces to Connect the Unconnectablecited by 70
  4. 2024Towards Analyzing and Understanding the Limitations of DPO: A Theoretical Perspectivecited by 69
  5. 2021Don't Generate Me: Training Differentially Private Generative Models with Sinkhorn Divergencecited by 54
  6. 2021Faster Single-loop Algorithms for Minimax Optimization without Strong Concavitycited by 47
  7. 2025Language Self-Play For Data-Free Trainingcited by 42
  8. 2022COLA: Consistent Learning with Opponent-Learning Awarenesscited by 38
  9. 2022Why neural networks find simple solutions: the many regularizers of geometric complexitycited by 38
  10. 2024Generative Artificial Intelligence: A Systematic Review and Applicationscited by 37
  11. 2022Semi-Parametric Neural Image Synthesiscited by 36
  12. 2023Free Lunch for Domain Adversarial Training: Environment Label Smoothingcited by 33
38 more
  1. 2021Near-optimal Local Convergence of Alternating Gradient Descent-Ascent for Minimax Optimizationcited by 31
  2. 2024PaGoDA: Progressive Growing of a One-Step Generator from a Low-Resolution Diffusion Teachercited by 27
  3. 2021Which priors matter? Benchmarking models for learning latent dynamicscited by 26
  4. 2021A Neural Tangent Kernel Perspective of GANscited by 21
  5. 2022Competitive Physics Informed Networkscited by 19
  6. 2021Deepfake Forensics via An Adversarial Gamecited by 15
  7. 2021Understanding Overparameterization in Generative Adversarial Networkscited by 14
  8. 2021Discretization Drift in Two-Player Gamescited by 14
  9. 2023SAN: Inducing Metrizability of GAN with Discriminative Normalized Linear Layercited by 14
  10. 2023Ten Years of Generative Adversarial Nets (GANs): A survey of the state-of-the-artcited by 14
  11. 2021Imitation by Predicting Observationscited by 13
  12. 2022Riemannian Hamiltonian methods for min-max optimization on manifoldscited by 12
  13. 2021Complex Momentum for Optimization in Gamescited by 11
  14. 2022Max-Min Off-Policy Actor-Critic Method Focusing on Worst-Case Robustness to Model Misspecificationcited by 11
  15. 2023On a continuous time model of gradient descent dynamics and instability in deep learningcited by 11
  16. 2021Convergence Rates of Two-Time-Scale Gradient Descent-Ascent Dynamics for Solving Nonconvex Min-Max Problemscited by 10
  17. 2022Towards Multi-Agent Reinforcement Learning driven Over-The-Counter Market Simulationscited by 10
  18. 2023Synthetic Health-related Longitudinal Data with Mixed-type Variables Generated using Diffusion Modelscited by 9
  19. 2025Negative Stepsizes Make Gradient-Descent-Ascent Convergecited by 9
  20. 2021Minimax Optimization with Smooth Algorithmic Adversariescited by 8
  21. 2021Nonconvex-Nonconcave Min-Max Optimization with a Small Maximization Domaincited by 8
  22. 2021Lyapunov Exponents for Diversity in Differentiable Gamescited by 8
  23. 2021The Geometric Occam's Razor Implicit in Deep Learningcited by 7
  24. 2022Domain Adversarial Training: A Game Perspectivecited by 7
  25. 2022Instability and Local Minima in GAN Training with Kernel Discriminatorscited by 7
  26. 2024Fast Nonlinear Two-Time-Scale Stochastic Approximation: Achieving O(1/k) Finite-Sample Complexitycited by 7
  27. 2024Aligning Individual and Collective Objectives in Multi-Agent Cooperationcited by 7
  28. 2021DO-GAN: A Double Oracle Framework for Generative Adversarial Networkscited by 6
  29. 2021Game of GANs: Game-Theoretical Models for Generative Adversarial Networkscited by 6
  30. 2022Deep Data Augmentation for Weed Recognition Enhancement: A Diffusion Probabilistic Model and Transfer Learning Based Approachcited by 6
  31. 2022Effects of Spectral Normalization in Multi-agent Reinforcement Learningcited by 6
  32. 2023Look ATME: The Discriminator Mean Entropy Needs Attentioncited by 6
  33. 2021Multi-View Feature Representation for Dialogue Generation with Bidirectional Distillationcited by 5
  34. 2021SyMetric: Measuring the Quality of Learnt Hamiltonian Dynamics Inferred from Visioncited by 5
  35. 2023Implicit Variational Inference for High-Dimensional Posteriorscited by 5
  36. 2023Peer is Your Pillar: A Data-unbalanced Conditional GANs for Few-shot Image Generationcited by 5
  37. 2025Multiplayer Federated Learning: Reaching Equilibrium with Less Communicationcited by 5
  38. 2021Functional Space Analysis of Local GAN Convergencecited by 4

50 most cited of 104 All 104 (JSON)