Polyak-Type Extragradient Methods for Monotone Root-Finding Problems
Not on the map. Why?
Abstract (arXiv)
We study Polyak-type step-size selection for extragradient methods for solving deterministic and stochastic monotone root-finding problems. We show that the known projection-type correction for deterministic extragradient arises from minimizing an upper bound on the distance to a solution, paralleling the classical Polyak step-size construction. Using this viewpoint, we provide a unified deterministic analysis of the Polyak-type Extragradient Method (PolyakEG), based on a local critical condition controlling the variation of operator F along the extrapolation direction. This analysis does not require global Lipschitz continuity, and covers sublinear convergence under broader conditions such as H"older continuity or (L0, L1)-Lipschitzness and linear convergence under additional strong monotonicity, all through a single framework. We then study the stochastic extensions of this approach. We first prove convergence of a direct stochastic variant, PolyakSEG, when all stochastic component operators share a common solution. We also show that, without this condition, PolyakSEG with nonvanishing step-sizes may fail to converge to a zero of the mean operator. To address this limitation, we propose DecPolyakSEG, which combines decreasing step-sizes with Polyak-type updates, and establish a sublinear residual convergence result without requiring a common solution across the component operators. These results parallel recent developments in stochastic Polyak step-sizes from the convex minimization literature and establish an analogous research avenue in the broader root-finding regime.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 62 of its 90 references that are in the map. First posted on arXiv 22 Sep 2026.
Builds on
62 of 90 references.
References · 62
- 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
- 2019Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithmscited by 555
- 2019Why gradient clipping accelerates training: A theoretical justification for adaptivitycited by 335
- 2013Optimization, Learning, and Games with Predictable Sequencescited by 187
- 2018A Variational Inequality Perspective on Generative Adversarial Networkscited by 144
- 2019A Unified Analysis of Extra-gradient and Optimistic Gradient Methods for Saddle Point Problems: Proximal Point Approachcited by 142
- 2020Stochastic Polyak Step-size for SGD: An Adaptive Learning Rate for Fast Convergencecited by 137
- 2018Optimistic mirror descent in saddle-point problems: Going the extra (gradient) milecited by 132
- 2019Painless Stochastic Gradient: Interpolation, Line-Search, and Convergence Ratescited by 119
- 2020Efficient Methods for Structured Nonconvex-Nonconcave Min-Max Optimizationcited by 94
- 2019On the convergence of single-call stochastic extra-gradient methodscited by 88
- 2021Accelerated Algorithms for Smooth Convex-Concave Minimax Problems with O(1/k2) Rate on Squared Gradient Normcited by 71
38 more
- 2017A First Order Method for Solving Convex Bi-Level Optimization Problemscited by 67
- 2020SGD for Structured Nonconvex Functions: Learning Rates, Minibatching and Interpolationcited by 66
- 2022A Unified Approach to Reinforcement Learning, Quantal Response Equilibria, and Two-Player Zero-Sum Gamescited by 64
- 2019Reducing Noise in GAN Training with Variance Reduced Extragradientcited by 61
- 2021Fast Extra Gradient Methods for Smooth Structured Nonconvex-Nonconcave Minimax Problemscited by 61
- 2020Halpern Iteration for Near-Optimal and Parameter-Free Monotone Inclusion and Strong Solutions to Variational Inequalitiescited by 60
- 2019Revisiting the Polyak step sizecited by 55
- 2020Explore Aggressively, Update Conservatively: Stochastic Extragradient Methods with Variable Stepsize Scalingcited by 49
- 2021Stochastic Gradient Descent-Ascent and Consensus Optimization for Smooth Games: Convergence Analysis under Expected Co-coercivitycited by 43
- 2022Dynamics of SGD with Stochastic Polyak Stepsizes: Truly Adaptive Variants and Convergence to Exact Solutioncited by 40
- 2021Extragradient Method: O(1/K) Last-Iterate Convergence for Monotone Variational Inequalities and Connections With Cocoercivitycited by 39
- 2019Revisiting Stochastic Extragradientcited by 35
- 2022Last-Iterate Convergence of Optimistic Gradient Method for Monotone Variational Inequalitiescited by 33
- 2020Stochastic Hamiltonian Gradient Methods for Smooth Gamescited by 32
- 2021Stochastic Extragradient: General Analysis and Improved Ratescited by 31
- 2022Accelerated Single-Call Methods for Constrained Min-Max Optimizationcited by 31
- 2020Adaptive extra-gradient methods for min-max optimization and gamescited by 30
- 2013On the Convergence Properties of Non-Euclidean Extragradient Methods for Variational Inequalities with Generalized Monotone Operatorscited by 29
- 2021Stochastic Mirror Descent: Convergence Analysis and Adaptive Variants via the Mirror Stochastic Polyak Stepsizecited by 22
- 2023Escaping limit cycles: Global convergence for constrained nonconvex-nonconcave minimax problemscited by 21
- 2021Halpern-Type Accelerated and Splitting Algorithms For Monotone Inclusionscited by 20
- 2014Optimal stochastic extragradient schemes for pseudomonotone stochastic variational inequality problems and their variantscited by 19
- 2021Stochastic Polyak Stepsize with a Moving Targetcited by 19
- 2022Near-Optimal Algorithms for Making the Gradient Small in Stochastic Minimax Optimizationcited by 19
- 2022From Halpern's Fixed-Point Iterations to Nesterov's Accelerated Interpretations for Root-Finding Problemscited by 18
- 2022Stochastic Halpern Iteration with Variance Reduction for Stochastic Monotone Inclusionscited by 17
- 2022Convergence of Proximal Point and Extragradient-Based Methods Beyond Monotonicity: the Case of Negative Comonotonicitycited by 16
- 2023Single-Call Stochastic Extragradient Methods for Structured Non-monotone Variational Inequalities: Improved Analysis under Weaker Conditionscited by 14
- 2024Stochastic Polyak Step-sizes and Momentum: Convergence Guarantees and Practical Performancecited by 13
- 2021Optimal error bounds for nonexpansive fixed-point iterations in normed spacescited by 12
- 2022Fast Optimistic Gradient Descent Ascent (OGDA) method in continuous and discrete timecited by 10
- 2023Variance Reduced Halpern Iteration for Finite-Sum Monotone Inclusionscited by 10
- 2022Accelerated Minimax Algorithms Flock Togethercited by 9
- 2023Randomized Block-Coordinate Optimistic Gradient Algorithms for Root-Finding Problemscited by 9
- 2024Optimal Acceleration for Minimax and Fixed-Point Problems is Not Uniquecited by 7
- 2016On stochastic mirror-prox algorithms for stochastic Cartesian variational inequalities: randomized block coordinate and optimal averaging schemescited by 5
- 2025Analysis of an Idealized Stochastic Polyak Method and its Application to Black-Box Model Distillationcited by 5
- 2019Nonlinear Forward-Backward Splitting with Projection Correctioncited by 4
50 most cited of 62 All 62 (JSON)
Built on by
Not cited yet.