mlsanityData to 7 Oct 2026

From Poincar'e Recurrence to Convergence in Imperfect Information Games: Finding Equilibrium via Regularization

arXiv 2002.08456 ↗First posted 19 Feb 2020cs.GT cs.LG stat.MLcited by 59

Not on the map. Why?

Abstract (arXiv)

In this paper we investigate the Follow the Regularized Leader dynamics in sequential imperfect information games (IIG). We generalize existing results of Poincar'e recurrence from normal-form games to zero-sum two-player imperfect information games and other sequential game settings. We then investigate how adapting the reward (by adding a regularization term) of the game can give strong convergence guarantees in monotone games. We continue by showing how this reward adaptation technique can be leveraged to build algorithms that converge exactly to the Nash equilibrium. Finally, we show how these insights can be directly used to build state-of-the-art model-free algorithms for zero-sum two-player Imperfect Information Games (IIG).

Key facts. Cited by 59 papers in the map (data through 7 Oct 2026). First posted on arXiv 19 Feb 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 10; 2022: 15; 2023: 11; 2024: 10; 2025: 9; 2026: 4.

59 papers.

Sort:

By area:

  1. 2023Nash Learning from Human Feedbackcited by 220
  2. 2022A Unified Approach to Reinforcement Learning, Quantal Response Equilibria, and Two-Player Zero-Sum Gamescited by 64
  3. 2021Discovering Diverse Multi-Agent Strategic Behavior via Reward Randomizationcited by 35
  4. 2022Scalable Deep Reinforcement Learning Algorithms for Mean Field Gamescited by 33
  5. 2022Adversarial Policies Beat Superhuman Go AIscited by 29
  6. 2024A Survey on Self-play Methods in Reinforcement Learningcited by 26
  7. 2022ESCHER: Eschewing Importance Sampling in Games by Computing a History Value Function to Estimate Regretcited by 24
  8. 2022Policy Mirror Ascent for Efficient and Independent Learning in Mean Field Gamescited by 24
  9. 2021Student of Games: A unified learning algorithm for both perfect and imperfect information gamescited by 22
  10. 2022The Power of Regularization in Solving Extensive-Form Gamescited by 21
  11. 2021Exploration-Exploitation in Multi-Agent Competition: Convergence with Bounded Rationalitycited by 20
  12. 2024Magnetic Preference Optimization: Achieving Last-iterate Convergence for Language Model Alignmentcited by 20
38 more
  1. 2023ReLOAD: Reinforcement Learning with Optimistic Ascent-Descent for Last-Iterate Convergence in Constrained MDPscited by 19
  2. 2021Provably Efficient Policy Optimization for Two-Player Zero-Sum Markov Gamescited by 18
  3. 2025Reevaluating Policy Gradient Methods for Imperfect-Information Gamescited by 13
  4. 2021Complex Momentum for Optimization in Gamescited by 11
  5. 2021Learning in Matrix Games can be Arbitrarily Complexcited by 11
  6. 2022Self-Play PSRO: Toward Optimal Populations in Two-Player Zero-Sum Gamescited by 11
  7. 2021Online Learning in Periodic Zero-Sum Gamescited by 10
  8. 2024RL-CFR: Improving Action Abstraction for Imperfect Information Extensive-Form Games with Reinforcement Learningcited by 10
  9. 2022Provably Efficient Fictitious Play Policy Optimization for Zero-Sum Markov Games with Structured Transitionscited by 9
  10. 2022Last-Iterate Convergence with Full and Noisy Feedback in Two-Player Zero-Sum Gamescited by 9
  11. 2023Adaptively Perturbed Mirror Descent for Learning in Gamescited by 9
  12. 2021Solving Min-Max Optimization with Hidden Structure via Gradient Descent Ascentcited by 8
  13. 2021Search in Imperfect Information Gamescited by 8
  14. 2022Mutation-Driven Follow the Regularized Leader for Last-Iterate Convergence in Zero-Sum Gamescited by 8
  15. 2024COMAL: A Convergent Meta-Algorithm for Aligning LLMs with General Preferencescited by 8
  16. 2023Rethinking Adversarial Policies: A Generalized Attack Formulation and Provable Defense in RLcited by 7
  17. 2024Minimizing Weighted Counterfactual Regret with Optimistic Online Mirror Descentcited by 7
  18. 2023Abstracting Imperfect Information Away from Two-Player Zero-Sum Gamescited by 6
  19. 2023Fictitious Cross-Play: Learning Global Nash Equilibrium in Mixed Cooperative-Competitive Gamescited by 6
  20. 2024A Policy-Gradient Approach to Solving Imperfect-Information Games with Best-Iterate Convergencecited by 5
  21. 2025RSPO: Regularized Self-Play Alignment of Large Language Modelscited by 5
  22. 2022Nash, Conley, and Computation: Impossibility and Incompleteness in Game Dynamicscited by 4
  23. 2022Fast Convergence of Optimistic Gradient Ascent in Network Zero-Sum Extensive Form Gamescited by 4
  24. 2024Neural Population Learning beyond Symmetric Zero-sum Gamescited by 4
  25. 2021Online Optimization in Games via Control Theory: Connecting Regret, Passivity and Poincar'e Recurrencecited by 3
  26. 2022A Survey of Decision Making in Adversarial Gamescited by 3
  27. 2022Developing, Evaluating and Scaling Learning Agents in Multi-Agent Environmentscited by 2
  28. 2023The Update-Equivalence Framework for Decision-Time Planningcited by 2
  29. 2023JiangJun: Mastering Xiangqi by Tackling Non-Transitivity in Two-Player Zero-Sum Gamescited by 2
  30. 2023Minimax Exploiter: A Data Efficient Approach for Competitive Self-Playcited by 2
  31. 2026Which Nash Equilibrium? Solver-Dependent Selection on Zero-Sum Nash Polytopescited by 2
  32. 2022Unified Policy Optimization for Continuous-action Reinforcement Learning in Non-stationary Tasks and Gamescited by 1
  33. 2025On the Power of Perturbation under Sampling in Solving Extensive-Form Gamescited by 1
  34. 2025Faster Game Solving via Asymmetry of Step Sizescited by 1
  35. 2025Look-ahead Reasoning with a Learned Model in Imperfect Information Gamescited by 1
  36. 2025NashPG: A Policy Gradient Method with Iteratively Refined Regularization for Finding Nash Equilibriacited by 1
  37. 2026The Curvature Shadow: An Apparent Failure of Maximum-Entropy Equilibrium Selection is a Removable Artifactcited by 1
  38. 2023Game-Theoretic Robust Reinforcement Learning Handles Temporally-Coupled Perturbationscited by 0

50 most cited of 59 All 59 (JSON)