mlsanityData to 7 Oct 2026

Polyak-Type Extragradient Methods for Monotone Root-Finding Problems

arXiv 2609.26581 ↗First posted 22 Sep 2026math.OC cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

We study Polyak-type step-size selection for extragradient methods for solving deterministic and stochastic monotone root-finding problems. We show that the known projection-type correction for deterministic extragradient arises from minimizing an upper bound on the distance to a solution, paralleling the classical Polyak step-size construction. Using this viewpoint, we provide a unified deterministic analysis of the Polyak-type Extragradient Method (PolyakEG), based on a local critical condition controlling the variation of operator F along the extrapolation direction. This analysis does not require global Lipschitz continuity, and covers sublinear convergence under broader conditions such as H"older continuity or (L0, L1)-Lipschitzness and linear convergence under additional strong monotonicity, all through a single framework. We then study the stochastic extensions of this approach. We first prove convergence of a direct stochastic variant, PolyakSEG, when all stochastic component operators share a common solution. We also show that, without this condition, PolyakSEG with nonvanishing step-sizes may fail to converge to a zero of the mean operator. To address this limitation, we propose DecPolyakSEG, which combines decreasing step-sizes with Polyak-type updates, and establish a sublinear residual convergence result without requiring a common solution across the component operators. These results parallel recent developments in stochastic Polyak step-sizes from the convex minimization literature and establish an analogous research avenue in the broader root-finding regime.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 62 of its 90 references that are in the map. First posted on arXiv 22 Sep 2026.

Builds on

62 of 90 references.

References · 62

  1. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  2. 2019Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithmscited by 555
  3. 2019Why gradient clipping accelerates training: A theoretical justification for adaptivitycited by 335
  4. 2013Optimization, Learning, and Games with Predictable Sequencescited by 187
  5. 2018A Variational Inequality Perspective on Generative Adversarial Networkscited by 144
  6. 2019A Unified Analysis of Extra-gradient and Optimistic Gradient Methods for Saddle Point Problems: Proximal Point Approachcited by 142
  7. 2020Stochastic Polyak Step-size for SGD: An Adaptive Learning Rate for Fast Convergencecited by 137
  8. 2018Optimistic mirror descent in saddle-point problems: Going the extra (gradient) milecited by 132
  9. 2019Painless Stochastic Gradient: Interpolation, Line-Search, and Convergence Ratescited by 119
  10. 2020Efficient Methods for Structured Nonconvex-Nonconcave Min-Max Optimizationcited by 94
  11. 2019On the convergence of single-call stochastic extra-gradient methodscited by 88
  12. 2021Accelerated Algorithms for Smooth Convex-Concave Minimax Problems with O(1/k2) Rate on Squared Gradient Normcited by 71
38 more
  1. 2017A First Order Method for Solving Convex Bi-Level Optimization Problemscited by 67
  2. 2020SGD for Structured Nonconvex Functions: Learning Rates, Minibatching and Interpolationcited by 66
  3. 2022A Unified Approach to Reinforcement Learning, Quantal Response Equilibria, and Two-Player Zero-Sum Gamescited by 64
  4. 2019Reducing Noise in GAN Training with Variance Reduced Extragradientcited by 61
  5. 2021Fast Extra Gradient Methods for Smooth Structured Nonconvex-Nonconcave Minimax Problemscited by 61
  6. 2020Halpern Iteration for Near-Optimal and Parameter-Free Monotone Inclusion and Strong Solutions to Variational Inequalitiescited by 60
  7. 2019Revisiting the Polyak step sizecited by 55
  8. 2020Explore Aggressively, Update Conservatively: Stochastic Extragradient Methods with Variable Stepsize Scalingcited by 49
  9. 2021Stochastic Gradient Descent-Ascent and Consensus Optimization for Smooth Games: Convergence Analysis under Expected Co-coercivitycited by 43
  10. 2022Dynamics of SGD with Stochastic Polyak Stepsizes: Truly Adaptive Variants and Convergence to Exact Solutioncited by 40
  11. 2021Extragradient Method: O(1/K) Last-Iterate Convergence for Monotone Variational Inequalities and Connections With Cocoercivitycited by 39
  12. 2019Revisiting Stochastic Extragradientcited by 35
  13. 2022Last-Iterate Convergence of Optimistic Gradient Method for Monotone Variational Inequalitiescited by 33
  14. 2020Stochastic Hamiltonian Gradient Methods for Smooth Gamescited by 32
  15. 2021Stochastic Extragradient: General Analysis and Improved Ratescited by 31
  16. 2022Accelerated Single-Call Methods for Constrained Min-Max Optimizationcited by 31
  17. 2020Adaptive extra-gradient methods for min-max optimization and gamescited by 30
  18. 2013On the Convergence Properties of Non-Euclidean Extragradient Methods for Variational Inequalities with Generalized Monotone Operatorscited by 29
  19. 2021Stochastic Mirror Descent: Convergence Analysis and Adaptive Variants via the Mirror Stochastic Polyak Stepsizecited by 22
  20. 2023Escaping limit cycles: Global convergence for constrained nonconvex-nonconcave minimax problemscited by 21
  21. 2021Halpern-Type Accelerated and Splitting Algorithms For Monotone Inclusionscited by 20
  22. 2014Optimal stochastic extragradient schemes for pseudomonotone stochastic variational inequality problems and their variantscited by 19
  23. 2021Stochastic Polyak Stepsize with a Moving Targetcited by 19
  24. 2022Near-Optimal Algorithms for Making the Gradient Small in Stochastic Minimax Optimizationcited by 19
  25. 2022From Halpern's Fixed-Point Iterations to Nesterov's Accelerated Interpretations for Root-Finding Problemscited by 18
  26. 2022Stochastic Halpern Iteration with Variance Reduction for Stochastic Monotone Inclusionscited by 17
  27. 2022Convergence of Proximal Point and Extragradient-Based Methods Beyond Monotonicity: the Case of Negative Comonotonicitycited by 16
  28. 2023Single-Call Stochastic Extragradient Methods for Structured Non-monotone Variational Inequalities: Improved Analysis under Weaker Conditionscited by 14
  29. 2024Stochastic Polyak Step-sizes and Momentum: Convergence Guarantees and Practical Performancecited by 13
  30. 2021Optimal error bounds for nonexpansive fixed-point iterations in normed spacescited by 12
  31. 2022Fast Optimistic Gradient Descent Ascent (OGDA) method in continuous and discrete timecited by 10
  32. 2023Variance Reduced Halpern Iteration for Finite-Sum Monotone Inclusionscited by 10
  33. 2022Accelerated Minimax Algorithms Flock Togethercited by 9
  34. 2023Randomized Block-Coordinate Optimistic Gradient Algorithms for Root-Finding Problemscited by 9
  35. 2024Optimal Acceleration for Minimax and Fixed-Point Problems is Not Uniquecited by 7
  36. 2016On stochastic mirror-prox algorithms for stochastic Cartesian variational inequalities: randomized block coordinate and optimal averaging schemescited by 5
  37. 2025Analysis of an Idealized Stochastic Polyak Method and its Application to Black-Box Model Distillationcited by 5
  38. 2019Nonlinear Forward-Backward Splitting with Projection Correctioncited by 4

50 most cited of 62 All 62 (JSON)

Built on by

Not cited yet.