Matching Multi-Loop Complexities with a Single Loop: Optimal Optimization Stationarity and Best-Known Game Stationarity in Nonconvex--Concave Minimax Optimization
Not on the map. Why?
Abstract (arXiv)
We introduce a new single-loop algorithmic framework for smooth nonconvex--concave minimax optimization. The resulting projected damped extragradient method combines projected extragradient updates, dual momentum, and a moving proximal center. Under both the optimization-stationarity and game-stationarity criteria, our method achieves the best-known complexity among single-loop first-order methods. For optimization stationarity, our method achieves a gradient complexity of O(L2DYbarDelta0varepsilon-3), where L is the gradient Lipschitz constant, DY bounds the diameter of the dual feasible set, and barDelta0 is an initialization quantity involving the value-function gap and the initial gradients. Moreover, by incorporating a fixed-center warm-up phase, the complexity can be improved to O(L2DYDeltaφvarepsilon-3), up to an additive lower-order cost, where Deltaφ:=φ(x0)-infxφ(x). We further establish a lower bound of Omega(L2DYDeltaφvarepsilon-3) for optimization stationarity over projected zero-respecting first-order methods. This lower bound proves that the warm-started version of our algorithm is optimal up to a constant factor for optimization stationarity within this oracle class. For game stationarity, our method achieves O!(L3/2DY1/2Deltaφvarepsilon-5/2) gradient complexity. This matches the best-known complexity of multi-loop first-order methods, thereby establishing the same complexity with a single-loop algorithmic structure. Under dual strong concavity, the proposed framework achieves O!(√kappa,LDeltaφvarepsilon-2) leading complexity for both stationarity criteria, where kappa=L/μ is the dual condition number, up to an additive initialization cost. The varepsilon-2 accuracy dependence is optimal under fixed regularity and initialization bounds.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 13 of its 14 references that are in the map. First posted on arXiv 16 Sep 2026.
Builds on
13 of 14 references.
References · 13
- 2019Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalizationcited by 1,247
- 2019On Gradient Descent Ascent for Nonconvex-Concave Minimax Problemscited by 270
- 2017Lower Bounds for Finding Stationary Points Icited by 126
- 2020Near-Optimal Algorithms for Minimax Optimizationcited by 119
- 2019Efficient Algorithms for Smooth Minimax Optimizationcited by 76
- 2018Weakly-Convex Concave Min-Max Optimization: Provable Algorithms and Applications in Machine Learningcited by 63
- 2020A Unified Single-loop Alternating Gradient Projection Algorithm for Nonconvex-Concave and Convex-Nonconcave Minimax Problemscited by 57
- 2017Learning with Average Top-k Losscited by 52
- 2021Faster Single-loop Algorithms for Minimax Optimization without Strong Concavitycited by 49
- 2021The Complexity of Nonconvex-Strongly-Concave Minimax Optimizationcited by 45
- 2021Complexity Lower Bounds for Nonconvex-Strongly-Concave Min-Max Optimizationcited by 36
- 2024Two-Timescale Gradient Descent Ascent Algorithms for Nonconvex Minimax Optimizationcited by 16
Built on by
Not cited yet.