mlsanityData to 14 Oct 2026

Convergence of GANs Training: A Game and Stochastic Control Methodology

arXiv 2112.00222 ↗First posted 1 Dec 2021stat.ML cs.LGcited by 2

Not on the map. Why?

Abstract (arXiv)

Training generative adversarial networks (GANs) is known to be difficult, especially for financial time series. This paper first analyzes the well-posedness problem in GANs minimax games and the convexity issue in GANs objective functions. It then proposes a stochastic control framework for hyper-parameters tuning in GANs training. The weak form of dynamic programming principle and the uniqueness and the existence of the value function in the viscosity sense for the corresponding minimax game are established. In particular, explicit forms for the optimal adaptive learning rate and batch size are derived and are shown to depend on the convexity of the objective function, revealing a relation between improper choices of learning rate and explosion in GANs training. Finally, empirical studies demonstrate that training algorithms incorporating this adaptive control approach outperform the standard ADAM method in terms of convergence and robustness. From GANs training perspective, the analysis in this paper provides analytical support for the popular practice of ``clipping'', and suggests that the convexity and well-posedness issues in GANs may be tackled through appropriate choices of hyper-parameters.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 38 of its 78 references that are in the map. First posted on arXiv 1 Dec 2021.

Builds on

38 of 78 references.

References · 38

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2016Improved Techniques for Training GANscited by 2,970
  3. 2017Improved Training of Wasserstein GANscited by 2,235
  4. 2014Conditional Generative Adversarial Netscited by 2,162
  5. 2015Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networkscited by 2,054
  6. 2016Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Networkcited by 1,908
  7. 2016An overview of gradient descent optimization algorithmscited by 1,033
  8. 2016Generative Adversarial Text to Image Synthesiscited by 735
  9. 2013Stochastic First- and Zeroth-order Methods for Nonconvex Stochastic Programmingcited by 724
  10. 2018Which Training Methods for GANs do actually Converge?cited by 679
  11. 2018Gradient Descent Finds Global Minima of Deep Neural Networkscited by 574
  12. 2017Towards Principled Methods for Training Generative Adversarial Networkscited by 464
26 more
  1. 2018On the Global Convergence of Gradient Descent for Over-parameterized Models using Optimal Transportcited by 426
  2. 2016Generating Videos with Scene Dynamicscited by 376
  3. 2016Generative Visual Manipulation on the Natural Image Manifoldcited by 348
  4. 2015Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networkscited by 214
  5. 2018Stochastic Modified Equations and Dynamics of Stochastic Gradient Algorithms I: Mathematical Foundationscited by 114
  6. 2019Quant GANs: Deep Generation of Financial Time Seriescited by 108
  7. 2013Accelerated Gradient Methods for Nonconvex Nonlinear and Stochastic Programmingcited by 75
  8. 2016Semantic Segmentation using Adversarial Networkscited by 73
  9. 2018Finding Mixed Nash Equilibria of Generative Adversarial Networkscited by 49
  10. 2020Robust Reinforcement Learning via Adversarial training with Langevin Dynamicscited by 45
  11. 2020Generating Realistic Stock Market Order Streamscited by 30
  12. 2020A mean-field analysis of two-player zero-sum gamescited by 28
  13. 2019A Closer Look at the Optimization Landscapes of Generative Adversarial Networkscited by 25
  14. 2019Deep Hedging: Learning to Simulate Equity Option Marketscited by 25
  15. 2019Time Series Simulation by Conditional Generative Adversarial Netcited by 16
  16. 2019CorrGAN: Sampling Realistic Financial Correlation Matrices Using Generative Adversarial Networkscited by 16
  17. 2020Using generative adversarial networks to synthesize artificial financial datasetscited by 10
  18. 2020Game on Random Environment, Mean-field Langevin System and Neural Networkscited by 8
  19. 2021Minimax Optimization with Smooth Algorithmic Adversariescited by 8
  20. 2021Tail of Distribution GAN (TailGAN): Generative-Adversarial-Network-Based Boundary Formationcited by 6
  21. 2016Contextual RNN-GANs for Abstract Reasoning Diagram Generationcited by 4
  22. 2019Improving reinforcement learning algorithms: towards optimal learning rate policiescited by 2
  23. 2021Generative Adversarial Network: Some Analytical Perspectivescited by 2
  24. 2013Stochastic Perron's method and elementary strategies for zero-sum differential gamescited by 1
  25. 2013On martingale problems with continuous-time mixing and values of zero-sum games without Isaacs conditioncited by 1
  26. 2020SDE approximations of GANs training and its long-run behaviorcited by 1

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 0; 2024: 1.

2 papers.

  1. 2024On the Nonconvexity of Push-Forward Constraints and Its Consequences in Machine Learningcited by 1
  2. 2022Large deviations for interacting particle dynamics for finding mixed equilibria in zero-sum gamescited by 0