mlsanityData to 14 Oct 2026

Mutation-Driven Follow the Regularized Leader for Last-Iterate Convergence in Zero-Sum Games

arXiv 2206.09254 ↗First posted 18 Jun 2022cs.GT cs.LG stat.MLcited by 8

Not on the map. Why?

Abstract (arXiv)

In this study, we consider a variant of the Follow the Regularized Leader (FTRL) dynamics in two-player zero-sum games. FTRL is guaranteed to converge to a Nash equilibrium when time-averaging the strategies, while a lot of variants suffer from the issue of limit cycling behavior, i.e., lack the last-iterate convergence guarantee. To this end, we propose mutant FTRL (M-FTRL), an algorithm that introduces mutation for the perturbation of action probabilities. We then investigate the continuous-time dynamics of M-FTRL and provide the strong convergence guarantees toward stationary points that approximate Nash equilibria under full-information feedback. Furthermore, our simulation demonstrates that M-FTRL can enjoy faster convergence rates than FTRL and optimistic FTRL under full-information feedback and surprisingly exhibits clear convergence under bandit feedback.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 16 of its 45 references that are in the map. First posted on arXiv 18 Jun 2022.

Builds on

16 of 45 references.

References · 16

  1. 2022Generative Adversarial Networkscited by 5,627
  2. 2017A Unified Game-Theoretic Approach to Multiagent Reinforcement Learningcited by 268
  3. 2017Training GANs with Optimismcited by 217
  4. 2013Optimization, Learning, and Games with Predictable Sequencescited by 188
  5. 2015Fast Convergence of Regularized Learning in Gamescited by 150
  6. 2018Optimistic mirror descent in saddle-point problems: Going the extra (gradient) milecited by 136
  7. 2017Cycles in adversarial regularized learningcited by 124
  8. 2018More Adaptive Algorithms for Adversarial Banditscited by 114
  9. 2018Last-Iterate Convergence: Zero-Sum Games and Constrained Min-Max Optimizationcited by 98
  10. 2014Solving Large Imperfect Information Games Using CFR+cited by 78
  11. 2018Solving Imperfect-Information Games via Discounted Regret Minimizationcited by 68
  12. 2020From Poincar'e Recurrence to Convergence in Imperfect Information Games: Finding Equilibrium via Regularizationcited by 61
4 more
  1. 2019Computing Approximate Equilibria in Sequential Adversarial Games by Exploitability Descentcited by 31
  2. 2018Variance Reduction in Monte Carlo Counterfactual Regret Minimization (VR-MCCFR) for Extensive Form Games using Baselinescited by 25
  3. 2020Approximate exploitability: Learning a best response in large gamescited by 23
  4. 2020Last iterate convergence in no-regret learning: constrained min-max optimization for convex-concave landscapescited by 21

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 2; 2024: 1; 2025: 4.

8 papers.

  1. 2025Amulet: ReAlignment During Test Time for Personalized Preference Adaptation of LLMscited by 29
  2. 2022Last-Iterate Convergence with Full and Noisy Feedback in Two-Player Zero-Sum Gamescited by 9
  3. 2023Adaptively Perturbed Mirror Descent for Learning in Gamescited by 9
  4. 2023Asymptotic Convergence and Performance of Multi-Agent Q-Learning Dynamicscited by 8
  5. 2025From Average-Iterate to Last-Iterate Convergence in Games: A Reduction and Its Applicationscited by 8
  6. 2024A Fisher-Rao gradient flow for entropic mean-field min-max gamescited by 3
  7. 2025Convergence and Connectivity: Dynamics of Multi-Agent Q-Learning in Random Networkscited by 1
  8. 2025Efficient Last-Iterate Convergence in Regret Minimization via Adaptive Reward Transformationcited by 0