mlsanityData to 14 Oct 2026

On Solving Minimax Optimization Locally: A Follow-the-Ridge Approach

arXiv 1910.07512 ↗First posted 16 Oct 2019cs.LG math.OC stat.MLcited by 34

Not on the map. Why?

Abstract (arXiv)

Many tasks in modern machine learning can be formulated as finding equilibria in sequential games. In particular, two-player zero-sum sequential games, also known as minimax optimization, have received growing interest. It is tempting to apply gradient descent to solve minimax optimization given its popularity and success in supervised learning. However, it has been noted that naive application of gradient descent fails to find some local minimax and can converge to non-local-minimax points. In this paper, we propose Follow-the-Ridge (FR), a novel algorithm that provably converges to and only converges to local minimax. We show theoretically that the algorithm addresses the notorious rotational behaviour of gradient dynamics, and is compatible with preconditioning and positive momentum. Empirically, FR solves toy minimax problems and improves the convergence of GAN training compared to the recent minimax optimization algorithms.

Key facts. Cited by 34 papers in the map (data through 14 Oct 2026). First posted on arXiv 16 Oct 2019.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 19; 2022: 9; 2023: 3; 2024: 2; 2025: 1.

34 papers.

Sort:

By area:

  1. 2021Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Modelscited by 252
  2. 2021Faster Single-loop Algorithms for Minimax Optimization without Strong Concavitycited by 49
  3. 2021Local Stochastic Gradient Descent Ascent: Convergence Analysis and Communication Efficiencycited by 40
  4. 2021Complexity Lower Bounds for Nonconvex-Strongly-Concave Min-Max Optimizationcited by 36
  5. 2021Learning to Give Checkable Answers with Prover-Verifier Gamescited by 32
  6. 2021Near-optimal Local Convergence of Alternating Gradient Descent-Ascent for Minimax Optimizationcited by 31
  7. 2022Indiscriminate Data Poisoning Attacks on Neural Networkscited by 25
  8. 2022Meta-learning Adaptive Deep Kernel Gaussian Processes for Molecular Property Predictioncited by 21
  9. 2022Multi-Rate VAE: Train Once, Get the Full Rate-Distortion Curvecited by 18
  10. 2021Discretization Drift in Two-Player Gamescited by 14
  11. 2021Finding Second-Order Stationary Points in Nonconvex-Strongly-Concave Minimax Optimizationcited by 12
  12. 2022Universal Gradient Descent Ascent Method for Nonconvex-Nonconcave Minimax Optimizationcited by 12
22 more
  1. 2021Complex Momentum for Optimization in Gamescited by 11
  2. 2021Minimax Optimization with Smooth Algorithmic Adversariescited by 8
  3. 2021Nonconvex-Nonconcave Min-Max Optimization with a Small Maximization Domaincited by 8
  4. 2021Last-Iterate Convergence of Saddle-Point Optimizers via High-Resolution Differential Equationscited by 8
  5. 2021Lyapunov Exponents for Diversity in Differentiable Gamescited by 8
  6. 2021A Cubic Regularization Approach for Finding Local Minimax Points in Nonconvex Minimax Optimizationcited by 5
  7. 2024Calibrated Dataset Condensation for Faster Hyperparameter Searchcited by 5
  8. 2021Learning for Robust Combinatorial Optimization: Algorithm and Applicationcited by 4
  9. 2021Accelerated Proximal Alternating Gradient-Descent-Ascent for Nonconvex Minimax Machine Learningcited by 4
  10. 2022STay-ON-the-Ridge: Guaranteed Convergence to Local Minimax Equilibrium in Nonconvex-Nonconcave Gamescited by 4
  11. 2023Two-timescale Extragradient for Finding Local Minimax Pointscited by 4
  12. 2021Generative Minimization Networks: Training GANs Without Competitioncited by 3
  13. 2022Stability and Generalization of Differentially Private Minimax Problemscited by 2
  14. 2022Recursive Reasoning in Minimax Games: A Level k Gradient Play Methodcited by 2
  15. 2021Exploiting the Hidden Tasks of GANs: Making Implicit Subproblems Explicitcited by 1
  16. 2022HessianFR: An Efficient Hessian-based Follow-the-Ridge Algorithm for Minimax Optimizationcited by 1
  17. 2023ApproxED: Approximate exploitability descent via learned best responsescited by 1
  18. 2023Distributionally Safe Reinforcement Learning under Model Uncertainty: A Single-Level Approach by Differentiable Convex Programmingcited by 1
  19. 2024Local convergence of simultaneous min-max algorithms to differential equilibrium on Riemannian manifoldcited by 1
  20. 2021GDA-AM: On the effectiveness of solving minimax optimization via Anderson Accelerationcited by 0
  21. 2022On the Nash equilibrium of moment-matching GANs for stationary Gaussian processescited by 0
  22. 2025Generative modeling using evolved quantum Boltzmann machinescited by 0