mlsanityData to 13 Oct 2026

Non-Expansive Two-Time-Scale Stochastic Approximation: A Fixed-Schedule One-Quarter Barrier and Bias-Corrected Acceleration

arXiv 2607.13414 ↗First posted 15 Jul 2026stat.ML cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Non-expansive two-time-scale stochastic approximation is governed by a slow stochastic Krasnoselskii--Mann fixed-point iteration rather than by contraction to a unique equilibrium. We study this regime under a contractive fast map and a non-expansive reduced slow map. We first prove a finite-horizon lower bound showing that, for any prescribed slow stepsize schedule (βk), the classical KM residual scale (sumi<Nβi(1-βi))-1 is worst-case sharp for the corresponding unregularized KM update. Combined with the raw fast-tracking leakage scale, this explains the previously observed k-1/4+o(1) last-iterate mean-square residual exponent. We then introduce a residual-preconditioned slow oracle that cancels the first-order dependence on the fast tracking error. In a nested Tikhonov-KM algorithm, the uncorrected oracle yields total-sample rate T-1/4+o(1), while the corrected oracle yields T-1/3+o(1). This improvement comes from changing the slow-oracle bias from first order to second order in the fast error after all inner-loop samples are counted. Finally, we show that the repeated inner-loop cost of the nested method can be avoided in a smooth derivative-oracle model. A single-loop algorithm that tracks both the fast equilibrium and the leakage preconditioner online achieves T-1/2+o(1) with O(1) primitive samples per iteration.

Key facts. No citing papers in the map yet (data through 13 Oct 2026). Builds on 14 of its 36 references that are in the map. First posted on arXiv 15 Jul 2026.

Builds on

14 of 36 references.

References · 14

  1. 2015Automatic differentiation in machine learning: a surveycited by 852
  2. 2020Bilevel Optimization: Convergence Analysis and Enhanced Designcited by 219
  3. 2020A Two-Timescale Framework for Bilevel Optimization: Complexity Analysis and Application to Actor-Criticcited by 141
  4. 2023A Fully First-Order Method for Stochastic Bilevel Optimizationcited by 97
  5. 2021Amortized Implicit Differentiation for Stochastic Bilevel Optimizationcited by 63
  6. 2020Finite Time Analysis of Linear Two-timescale Stochastic Approximation with Markovian Noisecited by 52
  7. 2020Nonlinear Two-Time-Scale Stochastic Approximation: Convergence and Finite-Time Performancecited by 35
  8. 2022Stochastic fixed-point iterations for nonexpansive maps: Convergence and error boundscited by 23
  9. 2024A Sharper Global Convergence Analysis for Average Reward Reinforcement Learning via an Actor-Critic Approachcited by 15
  10. 2024Stochastic Halpern iteration in normed spaces and applications to reinforcement learningcited by 11
  11. 2025On The Sample Complexity Bounds In Bilevel Reinforcement Learningcited by 10
  12. 2026Stochastic Krasnoselskii-Mann Iterations: Convergence without Uniformly Bounded Variancecited by 5
2 more
  1. 2025Non-Expansive Mappings in Two-Time-Scale Stochastic Approximation: Finite-Time Analysiscited by 3
  2. 2026On the Convergence of Self-Improving Online LLM Alignmentcited by 1

Built on by

Not cited yet.