mlsanityData to 7 Oct 2026

GDA-AM: On the effectiveness of solving minimax optimization via Anderson Acceleration

arXiv 2110.02457 ↗First posted 6 Oct 2021cs.LG cs.NA math.NAcited by 0

Not on the map. Why?

Abstract (arXiv)

Many modern machine learning algorithms such as generative adversarial networks (GANs) and adversarial training can be formulated as minimax optimization. Gradient descent ascent (GDA) is the most commonly used algorithm due to its simplicity. However, GDA can converge to non-optimal minimax points. We propose a new minimax optimization framework, GDA-AM, that views the GDAdynamics as a fixed-point iteration and solves it using Anderson Mixing to con-verge to the local minimax. It addresses the diverging issue of simultaneous GDAand accelerates the convergence of alternating GDA. We show theoretically that the algorithm can achieve global convergence for bilinear problems under mild conditions. We also empirically show that GDA-AMsolves a variety of minimax problems and improves GAN training on several datasets

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 36 of its 64 references that are in the map. First posted on arXiv 6 Oct 2021.

Builds on

36 of 64 references.

References · 36

  1. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  2. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  3. 2014Deep Learning Face Attributes in the Wildcited by 3,679
  4. 2016Improved Techniques for Training GANscited by 2,938
  5. 2017Improved Training of Wasserstein GANscited by 2,223
  6. 2018Spectral Normalization for Generative Adversarial Networkscited by 1,618
  7. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,510
  8. 2016Adversarial Machine Learning at Scalecited by 934
  9. 2019On Gradient Descent Ascent for Nonconvex-Concave Minimax Problemscited by 266
  10. 2017Training GANs with Optimismcited by 217
  11. 2019Solving a Class of Non-Convex Min-Max Games Using Iterative First Order Methodscited by 148
  12. 2018A Variational Inequality Perspective on Generative Adversarial Networkscited by 144
24 more
  1. 2019A Unified Analysis of Extra-gradient and Optimistic Gradient Methods for Saddle Point Problems: Proximal Point Approachcited by 142
  2. 2018Optimistic mirror descent in saddle-point problems: Going the extra (gradient) milecited by 132
  3. 2017The Numerics of GANscited by 104
  4. 2019On the convergence of single-call stochastic extra-gradient methodscited by 88
  5. 2019Efficient Algorithms for Smooth Minimax Optimizationcited by 75
  6. 2020Stochastic Recursive Gradient Descent Ascent for Stochastic Nonconvex-Strongly-Concave Minimax Problemscited by 75
  7. 2018The Unusual Effectiveness of Averaging in GAN Trainingcited by 73
  8. 2018Negative Momentum for Improved Game Dynamicscited by 59
  9. 2020A Single-Loop Smoothed Gradient Descent-Ascent Algorithm for Nonconvex-Concave Min-Max Problemscited by 57
  10. 2020A Unified Single-loop Alternating Gradient Projection Algorithm for Nonconvex-Concave and Convex-Nonconcave Minimax Problemscited by 56
  11. 2020Efficient Search of First-Order Nash Equilibria in Nonconvex-Concave Smooth Min-Max Problemscited by 39
  12. 2019On Finding Local Nash Equilibria (and Only Local Nash Equilibria) in Zero-Sum Gamescited by 35
  13. 2019On Solving Minimax Optimization Locally: A Follow-the-Ridge Approachcited by 34
  14. 2019Competitive Gradient Descentcited by 32
  15. 2021Near-optimal Local Convergence of Alternating Gradient Descent-Ascent for Minimax Optimizationcited by 31
  16. 2018Local Saddle Point Optimization: A Curvature Exploitation Approachcited by 28
  17. 2020Last iterate convergence in no-regret learning: constrained min-max optimization for convex-concave landscapescited by 21
  18. 2020Ridge Rider: Finding Diverse Solutions by Following Eigenvectors of the Hessiancited by 20
  19. 2020Accelerating Smooth Games by Manipulating Spectral Shapescited by 18
  20. 2020Improving GAN Training with Probability Ratio Clipping and Sample Reweightingcited by 18
  21. 2020On the Suboptimality of Negative Momentum for Minimax Optimizationcited by 11
  22. 2019A Stochastic Extra-Step Quasi-Newton Method for Nonsmooth Nonconvex Optimizationcited by 7
  23. 2021Stochastic Anderson Mixing for Nonconvex Stochastic Optimizationcited by 6
  24. 2018Nonlinear Acceleration of Momentum and Primal-Dual Algorithmscited by 4

Built on by

Not cited yet.