# Student of Games: A unified learning algorithm for both perfect and imperfect information games

arXiv: https://arxiv.org/abs/2112.03178 · HTML: https://mlsanity.com/paper/2112.03178/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 76 references that are in the map.
- First posted on arXiv 6 Dec 2021.

## Builds on (most cited first, 27 of 27)

- [DeepStack: Expert-Level Artificial Intelligence in No-Limit Poker](https://mlsanity.com/paper/1701.01724/) (2017)
- [The Hanabi Challenge: A New Frontier for AI Research](https://mlsanity.com/paper/1902.00506/) (2019)
- [OpenSpiel: A Framework for Reinforcement Learning in Games](https://mlsanity.com/paper/1908.09453/) (2019)
- [Collaborating with Humans without Human Data](https://mlsanity.com/paper/2110.08176/) (2021)
- [Combining Deep Reinforcement Learning and Search for Imperfect-Information Games](https://mlsanity.com/paper/2007.13544/) (2020)
- [Deep Counterfactual Regret Minimization](https://mlsanity.com/paper/1811.00164/) (2018)
- [Bayesian Action Decoder for Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1811.01458/) (2018)
- [From Poincar'e Recurrence to Convergence in Imperfect Information Games: Finding Equilibrium via Regularization](https://mlsanity.com/paper/2002.08456/) (2020)
- [Safe and Nested Subgame Solving for Imperfect-Information Games](https://mlsanity.com/paper/1705.02955/) (2017)
- [Rethinking Formal Models of Partially Observable Multiagent Decision Making](https://mlsanity.com/paper/1906.11110/) (2019)
- [DREAM: Deep Regret minimization with Advantage baselines and Model-free learning](https://mlsanity.com/paper/2006.10410/) (2020)
- [Actor-Critic Policy Optimization in Partially Observable Multiagent Environments](https://mlsanity.com/paper/1810.09026/) (2018)
- [XDO: A Double Oracle Algorithm for Extensive-Form Games](https://mlsanity.com/paper/2103.06426/) (2021)
- [Improving Policies via Search in Cooperative Partially Observable Games](https://mlsanity.com/paper/1912.02318/) (2019)
- [Computing Approximate Equilibria in Sequential Adversarial Games by Exploitability Descent](https://mlsanity.com/paper/1903.05614/) (2019)
- [No-Press Diplomacy from Scratch](https://mlsanity.com/paper/2110.02924/) (2021)
- [Solving Imperfect Information Games Using Decomposition](https://mlsanity.com/paper/1303.4441/) (2013)
- [Learning to Play No-Press Diplomacy with Best Response Policy Iteration](https://mlsanity.com/paper/2006.04635/) (2020)
- [The Advantage Regret-Matching Actor-Critic](https://mlsanity.com/paper/2008.12234/) (2020)
- [Double Neural Counterfactual Regret Minimization](https://mlsanity.com/paper/1812.10607/) (2018)
- [Solving Common-Payoff Games with Approximate Policy Iteration](https://mlsanity.com/paper/2101.04237/) (2021)
- [AIVAT: A New Variance Reduction Technique for Agent Evaluation in Imperfect Information Games](https://mlsanity.com/paper/1612.06915/) (2016)
- [Unlocking the Potential of Deep Counterfactual Value Networks](https://mlsanity.com/paper/2007.10442/) (2020)
- [Search in Imperfect Information Games](https://mlsanity.com/paper/2111.05884/) (2021)
- [Monte Carlo Continual Resolving for Online Strategy Computation in Imperfect Information Games](https://mlsanity.com/paper/1812.07351/) (2018)
- [Human-Agent Cooperation in Bridge Bidding](https://mlsanity.com/paper/2011.14124/) (2020)
- [Sound Algorithms in Imperfect Information Games](https://mlsanity.com/paper/2006.08740/) (2020)

## Built on by (most cited first, 23 of 23)

- [Human vs. Machine: Behavioral Differences Between Expert Humans and Language Models in Wargame Simulations](https://mlsanity.com/paper/2403.03407/) (2024)
- [Sample-Efficient Learning of Correlated Equilibria in Extensive-Form Games](https://mlsanity.com/paper/2205.07223/) (2022)
- [General search techniques without common knowledge for imperfect-information games, and application to superhuman Fog of War chess](https://mlsanity.com/paper/2506.01242/) (2025)
- [Human-Level Competitive Pok'emon via Scalable Offline Reinforcement Learning with Transformers](https://mlsanity.com/paper/2504.04395/) (2025)
- [Cardiverse: Harnessing LLMs for Novel Card Game Prototyping](https://mlsanity.com/paper/2502.07128/) (2025)
- [Neural Bayesian Filtering](https://mlsanity.com/paper/2510.03614/) (2025)
- [Look-ahead Reasoning with a Learned Model in Imperfect Information Games](https://mlsanity.com/paper/2510.05048/) (2025)
- [Configurable Mirror Descent: Towards a Unification of Decision Making](https://mlsanity.com/paper/2405.11746/) (2024)
- [Modeling Uncertainty: Constraint-Based Belief States in Imperfect-Information Games](https://mlsanity.com/paper/2507.19263/) (2025)
- [AV-AIVAT: 74x Cheaper Agent Evaluation with Certified Anytime-Valid Stopping in Imperfect-Information Games](https://mlsanity.com/paper/2608.06362/) (2026)
- [AI planning in the imagination: High-level planning on learned abstract search spaces](https://mlsanity.com/paper/2308.08693/) (2023)
- [State-Constrained Zero-Sum Differential Games with One-Sided Information](https://mlsanity.com/paper/2403.02741/) (2024)
- [HSVI-based Online Minimax Strategies for Partially Observable Stochastic Games with Neural Perception Mechanisms](https://mlsanity.com/paper/2404.10679/) (2024)
- [GPU-Accelerated Counterfactual Regret Minimization](https://mlsanity.com/paper/2408.14778/) (2024)
- [Meta-Learning in Self-Play Regret Minimization](https://mlsanity.com/paper/2504.18917/) (2025)
- [Outbidding and Outbluffing Elite Humans: Mastering Liar's Poker via Self-Play and Reinforcement Learning](https://mlsanity.com/paper/2511.03724/) (2025)
- [SkyNet: Belief-Aware Planning for Partially-Observable Stochastic Games](https://mlsanity.com/paper/2603.27751/) (2026)
- [Self-Play Reinforcement Learning under Imperfect Information in Big 2](https://mlsanity.com/paper/2605.28863/) (2026)
- [Coachable agents for interactive gameplay](https://mlsanity.com/paper/2607.00642/) (2026)
- [PokaiTrainer: Scaling Equilibrium Search to Competitive Pok'emon VGC](https://mlsanity.com/paper/2608.29197/) (2026)
- [Test-time Reinforcement Learning in Imperfect Information Games](https://mlsanity.com/paper/2608.30635/) (2026)
- [Unsound Search with Policy and Value Networks in Legends of Code and Magic](https://mlsanity.com/paper/2609.06816/) (2026)
- [GPU-CFR: 80x Faster Counterfactual Regret Minimization by Compiling the Game to Static Dataflow and CUDA Graph Replay](https://mlsanity.com/paper/2609.11923/) (2026)

## Abstract (arXiv)

Games have a long history as benchmarks for progress in artificial intelligence. Approaches using search and learning produced strong performance across many perfect information games, and approaches using game-theoretic reasoning and learning demonstrated strong performance for specific imperfect information poker variants. We introduce Student of Games, a general-purpose algorithm that unifies previous approaches, combining guided search, self-play learning, and game-theoretic reasoning. Student of Games achieves strong empirical performance in large perfect and imperfect information games -- an important step towards truly general algorithms for arbitrary environments. We prove that Student of Games is sound, converging to perfect play as available computation and approximation capacity increases. Student of Games reaches strong performance in chess and Go, beats the strongest openly available agent in heads-up no-limit Texas hold'em poker, and defeats the state-of-the-art agent in Scotland Yard, an imperfect information game that illustrates the value of guided search, learning, and game-theoretic reasoning.