mlsanityData to 14 Oct 2026

Solving a class of non-convex min-max games using adaptive momentum methods

arXiv 2104.12676 ↗First posted 26 Apr 2021math.OC cs.LG stat.MLcited by 10

Not on the map. Why?

Abstract (arXiv)

Adaptive momentum methods have recently attracted a lot of attention for training of deep neural networks. They use an exponential moving average of past gradients of the objective function to update both search directions and learning rates. However, these methods are not suited for solving min-max optimization problems that arise in training generative adversarial networks. In this paper, we propose an adaptive momentum min-max algorithm that generalizes adaptive momentum methods to the non-convex min-max regime. Further, we establish non-asymptotic rates of convergence for the proposed algorithm when used in a reasonably broad class of non-convex min-max optimization problems. Experimental results illustrate its superior performance vis-a-vis benchmark methods for solving such problems.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Builds on 27 of its 42 references that are in the map. First posted on arXiv 26 Apr 2021.

Builds on

27 of 42 references.

References · 27

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,988
  3. 2016Improved Techniques for Training GANscited by 2,970
  4. 2016Optimization Methods for Large-Scale Machine Learningcited by 1,150
  5. 2019On the Convergence of Adam and Beyondcited by 841
  6. 2017Wasserstein GANcited by 639
  7. 2019On Gradient Descent Ascent for Nonconvex-Concave Minimax Problemscited by 270
  8. 2017Training GANs with Optimismcited by 217
  9. 2019SGD: General Analysis and Improved Ratescited by 207
  10. 2018On the Convergence of A Class of Adam-Type Algorithms for Non-Convex Optimizationcited by 174
  11. 2019Solving a Class of Non-Convex Min-Max Games Using Iterative First Order Methodscited by 148
  12. 2018Optimistic mirror descent in saddle-point problems: Going the extra (gradient) milecited by 136
15 more
  1. 2019Efficient Algorithms for Smooth Minimax Optimizationcited by 76
  2. 2019A Decentralized Parallel Algorithm for Training Generative Adversarial Netscited by 43
  3. 2020Efficient Search of First-Order Nash Equilibria in Nonconvex-Concave Smooth Min-Max Problemscited by 39
  4. 2020Non-convex Min-Max Optimization: Applications, Challenges, and Recent Theoretical Advancescited by 38
  5. 2017Extragradient method with variance reduction for stochastic variational inequalitiescited by 33
  6. 2019DADAM: A Consensus-based Distributed Adaptive Gradient Method for Online Optimizationcited by 30
  7. 2019Towards Better Understanding of Adaptive Gradient Algorithms in Generative Adversarial Netscited by 29
  8. 2019On the Convergence Proof of AMSGrad and a New Versioncited by 21
  9. 2019What is Local Optimality in Nonconvex-Nonconcave Minimax Optimization?cited by 20
  10. 2020GANs May Have No Nash Equilibriacited by 20
  11. 2016Frank-Wolfe Algorithms for Saddle Point Problemscited by 18
  12. 2020Solving Non-Convex Non-Differentiable Min-Max Games using Proximal Gradient Methodcited by 10
  13. 2020Adaptive First-and Zeroth-order Methods for Weakly Convex Stochastic Optimization Problemscited by 7
  14. 2018On the Behavior of the Expectation-Maximization Algorithm for Mixture Modelscited by 4
  15. 2019Training generative networks using random discriminatorscited by 4

Built on by

Cited by year

Citing papers per year: 2021: 5; 2022: 2; 2023: 1; 2024: 1; 2025: 1.

10 papers.

  1. 2022FedNest: Federated Bilevel, Minimax, and Compositional Optimizationcited by 60
  2. 2021Decentralized Local Stochastic Extra-Gradient for Variational Inequalitiescited by 25
  3. 2021A Decentralized Adaptive Momentum Method for Solving a Class of Min-Max Optimization Problemscited by 9
  4. 2021Nonconvex-Nonconcave Min-Max Optimization with a Small Maximization Domaincited by 8
  5. 2021On Characterizing GAN Convergence Through Proximal Duality Gapcited by 3
  6. 2022Dissecting adaptive methods in GANscited by 3
  7. 2025A Novel Multi-Task Teacher-Student Architecture with Self-Supervised Pretraining for 48-Hour Vasoactive-Inotropic Trend Analysis in Sepsis Mortality Predictioncited by 3
  8. 2024Accelerated Stochastic Min-Max Optimization Based on Bias-corrected Momentumcited by 2
  9. 2021Efficient Algorithms for Estimating the Parameters of Mixed Linear Regression Modelscited by 1
  10. 2023Federated Multi-Sequence Stochastic Approximation with Local Hypergradient Estimationcited by 0