mlsanityData to 13 Oct 2026

Mastering the Game of Stratego with Model-Free Multiagent Reinforcement Learning

arXiv 2206.15378 ↗First posted 30 Jun 2022cs.AI cs.GT cs.MAcited by 129

Not on the map. Why?

Cited by 28 in Deep reinforcement learning methods, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We introduce DeepNash, an autonomous agent capable of learning to play the imperfect information game Stratego from scratch, up to a human expert level. Stratego is one of the few iconic board games that Artificial Intelligence (AI) has not yet mastered. This popular game has an enormous game tree on the order of 10535 nodes, i.e., 10175 times larger than that of Go. It has the additional complexity of requiring decision-making under imperfect information, similar to Texas hold'em poker, which has a significantly smaller game tree (on the order of 10164 nodes). Decisions in Stratego are made over a large number of discrete actions with no obvious link between action and outcome. Episodes are long, with often hundreds of moves before a player wins, and situations in Stratego can not easily be broken down into manageably-sized sub-problems as in poker. For these reasons, Stratego has been a grand challenge for the field of AI for decades, and existing AI methods barely reach an amateur level of play. DeepNash uses a game-theoretic, model-free deep reinforcement learning method, without search, that learns to master Stratego via self-play. The Regularised Nash Dynamics (R-NaD) algorithm, a key component of DeepNash, converges to an approximate Nash equilibrium, instead of 'cycling' around it, by directly modifying the underlying multi-agent learning dynamics. DeepNash beats existing state-of-the-art AI methods in Stratego and achieved a yearly (2022) and all-time top-3 rank on the Gravon games platform, competing with human expert players.

Key facts. Cited by 129 papers in the map (data through 13 Oct 2026). Builds on 6 of its 33 references that are in the map. First posted on arXiv 30 Jun 2022.

Builds on

6 of 33 references.

References · 6

  1. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,093
  2. 2018Human-level performance in first-person multiplayer games with population-based deep reinforcement learningcited by 260
  3. 2017DeepStack: Expert-Level Artificial Intelligence in No-Limit Pokercited by 228
  4. 2019OpenSpiel: A Framework for Reinforcement Learning in Gamescited by 183
  5. 2021Podracer architectures for scalable Reinforcement Learningcited by 37
  6. 2013Measuring the Size of Large No-Limit Poker Gamescited by 14

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2022: 12; 2023: 36; 2024: 29; 2025: 28; 2026: 24.

129 papers.

Sort:

By area:

  1. 2023Nash Learning from Human Feedbackcited by 223
  2. 2023Harms from Increasingly Agentic Algorithmic Systemscited by 135
  3. 2023Avalon's Game of Thoughts: Battle Against Deception through Recursive Contemplationcited by 103
  4. 2024Open-Endedness is Essential for Artificial Superhuman Intelligencecited by 80
  5. 2025Robust Autonomy Emerges from Self-Playcited by 64
  6. 2024GPUDrive: Data-driven, multi-agent driving simulation at 1 million FPScited by 44
  7. 2023Honesty Is the Best Policy: Defining and Mitigating AI Deceptioncited by 40
  8. 2024RIME: Robust Preference-based Reinforcement Learning with Noisy Preferencescited by 37
  9. 2023The Consensus Game: Language Model Generation via Equilibrium Searchcited by 36
  10. 2023Brain-inspired Computational Intelligence via Predictive Codingcited by 30
  11. 2022Adversarial Policies Beat Superhuman Go AIscited by 29
  12. 2023Uncoupled and Convergent Learning in Two-Player Zero-Sum Markov Games with Bandit Feedbackcited by 29
38 more
  1. 2023ASP: Learn a Universal Neural Solver!cited by 27
  2. 2024A Survey on Self-play Methods in Reinforcement Learningcited by 26
  3. 2022Beyond CAGE: Investigating Generalization of Learned Autonomous Network Defense Policiescited by 22
  4. 2025DSGBench: A Diverse Strategic Game Benchmark for Evaluating LLM-based Agents in Complex Decision-Making Environmentscited by 22
  5. 2022General Intelligence Requires Rethinking Explorationcited by 20
  6. 2024Magnetic Preference Optimization: Achieving Last-iterate Convergence for Language Model Alignmentcited by 20
  7. 2025Building reliable sim driving agents by scaling self-playcited by 20
  8. 2024SwarmBrain: Embodied agent for real-time strategy game StarCraft II via large language modelscited by 19
  9. 2025Enhancing Rating-Based Reinforcement Learning to Effectively Leverage Feedback from Large Vision-Language Modelscited by 15
  10. 2024Fast Last-Iterate Convergence of Learning in Games Requires Forgetful Algorithmscited by 14
  11. 2024Adaptive Advantage-Guided Policy Regularization for Offline Reinforcement Learningcited by 13
  12. 2025Reevaluating Policy Gradient Methods for Imperfect-Information Gamescited by 13
  13. 2023Doubly Optimal No-Regret Learning in Monotone Gamescited by 12
  14. 2024PUZZLES: A Benchmark for Neural Algorithmic Reasoningcited by 12
  15. 2022Self-Play PSRO: Toward Optimal Populations in Two-Player Zero-Sum Gamescited by 11
  16. 2022Adapting to game trees in zero-sum imperfect information gamescited by 11
  17. 2023Mastering Strategy Card Game (Legends of Code and Magic) via End-to-End Policy and Optimistic Smooth Fictitious Playcited by 10
  18. 2023Hardness of Independent Learning and Sparse Equilibrium Computation in Markov Gamescited by 10
  19. 2023SRL: Scaling Distributed Reinforcement Learning to Over Ten Thousand Corescited by 10
  20. 2024Policy Space Response Oracles: A Surveycited by 10
  21. 2024PLAYER*: Enhancing LLM-based Multi-Agent Communication and Interaction in Murder Mystery Gamescited by 10
  22. 2024Offline-Boosted Actor-Critic: Adaptively Blending Optimal Historical Behaviors in Deep Off-Policy RLcited by 10
  23. 2023Towards practical reinforcement learning for tokamak magnetic controlcited by 9
  24. 2025Instance-Dependent Regret Bounds for Learning Two-Player Zero-Sum Games with Bandit Feedbackcited by 9
  25. 2024COMAL: A Convergent Meta-Algorithm for Aligning LLMs with General Preferencescited by 8
  26. 2025From Average-Iterate to Last-Iterate Convergence in Games: A Reduction and Its Applicationscited by 7
  27. 2025Scalable Decision Making for Games of Imperfect Informationcited by 7
  28. 2022Efficiently Computing Nash Equilibria in Adversarial Team Markov Gamescited by 6
  29. 2023Abstracting Imperfect Information Away from Two-Player Zero-Sum Gamescited by 6
  30. 2022Approximating Discontinuous Nash Equilibrial Values of Two-Player General-Sum Differential Gamescited by 5
  31. 2023Ensemble Value Functions for Efficient Exploration in Multi-Agent Reinforcement Learningcited by 5
  32. 2023Population-based Evaluation in Repeated Rock-Paper-Scissors as a Benchmark for Multiagent Reinforcement Learningcited by 5
  33. 2023An Invitation to Deep Reinforcement Learningcited by 5
  34. 2024A Policy-Gradient Approach to Solving Imperfect-Information Games with Best-Iterate Convergencecited by 5
  35. 2025RSPO: Regularized Self-Play Alignment of Large Language Modelscited by 5
  36. 2025General search techniques without common knowledge for imperfect-information games, and application to superhuman Fog of War chesscited by 5
  37. 2025Multi-Objective Reinforcement Learning with Max-Min Criterion: A Game-Theoretic Approachcited by 5
  38. 2024Symbolic Equation Solving via Reinforcement Learningcited by 4

50 most cited of 129 All 129 (JSON)