mlsanityData to 14 Oct 2026

Scalable Decision Making for Games of Imperfect Information

arXiv 2511.07312 ↗First posted 10 Nov 2025cs.LG cs.AIcited by 8

On the map: Deep reinforcement learning methods → RL for games and combinatorial optimization

Abstract (arXiv)

Real-world decision-making generally involves hidden information, that is, information that is unknown to one agent but possessed by another. Unfortunately, the presence of large amounts of hidden information renders established reinforcement learning and search approaches ineffective. Even with multimillion-dollar industrial research efforts, top-human-level play at Stratego---a board wargame with hidden information on a massive scale---has remained beyond the reach of artificial intelligence (AI). Here we introduce Ataraxos, an AI for Stratego based on general techniques that we developed for both self-play reinforcement learning and test-time search under hidden information. Ataraxos defeated the most decorated human Stratego player of all time by a large margin---achieving, to our knowledge, the first superhuman result in the game's history---while consuming orders of magnitude less compute and data than previous efforts. Using the same techniques, we built a superhuman AI for Barrage Stratego and state-of-the-art AIs for Hanabi and dou dizhu, all with low cost and high sample efficiency. The success of this approach across adversarial, cooperative and team games establishes a design pattern for reinforcement learning and search that is effective under large amounts of hidden information, a longstanding desideratum of the field of strategic decision-making.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Placed in RL for games and combinatorial optimization, one of 15 directions in Deep reinforcement learning methods. Builds on 23 of its 46 references that are in the map. First posted on arXiv 10 Nov 2025.

Builds on

23 of 46 references.

References · 23

  1. 2017Attention Is All You Needcited by 60,361
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  4. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 2,026
  5. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,353
  6. 2017Convolutional Sequence to Sequence Learningcited by 552
  7. 2017DeepStack: Expert-Level Artificial Intelligence in No-Limit Pokercited by 233
  8. 2019The Hanabi Challenge: A New Frontier for AI Researchcited by 197
  9. 2020Spatial-Temporal Transformer Networks for Traffic Flow Forecastingcited by 180
  10. 2022Mastering the Game of Stratego with Model-Free Multiagent Reinforcement Learningcited by 131
  11. 2020Combining Deep Reinforcement Learning and Search for Imperfect-Information Gamescited by 114
  12. 2025Robust Autonomy Emerges from Self-Playcited by 67
11 more
  1. 2022A Unified Approach to Reinforcement Learning, Quantal Response Equilibria, and Two-Player Zero-Sum Gamescited by 66
  2. 2019Accelerating Self-Play Learning in Gocited by 64
  3. 2017Safe and Nested Subgame Solving for Imperfect-Information Gamescited by 41
  4. 2018Depth-Limited Solving for Imperfect-Information Gamescited by 27
  5. 2021Scalable Online Planning via Reinforcement Learning Fine-Tuningcited by 18
  6. 2022PerfectDou: Dominating DouDizhu with Perfect Information Distillationcited by 15
  7. 2024Mastering Chess with a Transformer Modelcited by 15
  8. 2020Unlocking the Potential of Deep Counterfactual Value Networkscited by 8
  9. 2023Abstracting Imperfect Information Away from Two-Player Zero-Sum Gamescited by 6
  10. 2025General search techniques without common knowledge for imperfect-information games, and application to superhuman Fog of War chesscited by 6
  11. 2023The Update-Equivalence Framework for Decision-Time Planningcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 8.

8 papers.

  1. 2026Human-like autonomy emerges from self-play and a pinch of human datacited by 4
  2. 2026Data-Augmented Game Starts for Accelerating Self-Play Exploration in Imperfect Information Gamescited by 1
  3. 2026EMAgnet: Parameter-Space EMA Regularization for Policy Gradient Self-Play in Large Gamescited by 1
  4. 2026Two-Bridge: Exclusive Objectives and Extended Horizon StarCraft II Benchmarkcited by 0
  5. 2026GAE Falls Short in Imperfect-Information Self-Play Reinforcement Learningcited by 0
  6. 2026Superhuman AI for Generals.io Using Self-Play Reinforcement Learningcited by 0
  7. 2026Test-time Reinforcement Learning in Imperfect Information Gamescited by 0
  8. 2026NashDreamer: Model-Based Reinforcement Learning for Zero-Sum Imperfect-Information Gamescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Can AI Agents Learn Their Way to the Top? Evaluating Heuristic Learning in a Long-Running Game Agent Competition
  2. 2025Playing Hex and Counter Wargames using Reinforcement Learning and Recurrent Neural Networks
  3. 2025OptionZero: Planning with Learned Options
  4. 2025Reinforcement Learning in Strategy-Based and Atari Games: A Review of Google DeepMinds Innovations
  5. 2025Artificial Generals Intelligence: Mastering Generals.io with Reinforcement Learning
  6. 2026Mastering the Game of Go with Self-play Experience Replay
  7. 2021Modeling Strong and Human-Like Gameplay with KL-Regularized Search
  8. 2025Human-Level Competitive Pok'emon via Scalable Offline Reinforcement Learning with Transformers