mlsanityData to 7 Oct 2026

OpenSpiel: A Framework for Reinforcement Learning in Games

arXiv 1908.09453 ↗First posted 26 Aug 2019cs.LG cs.AI cs.GTcited by 180

Not on the map. Why?

Abstract (arXiv)

OpenSpiel is a collection of environments and algorithms for research in general reinforcement learning and search/planning in games. OpenSpiel supports n-player (single- and multi- agent) zero-sum, cooperative and general-sum, one-shot and sequential, strictly turn-taking and simultaneous-move, perfect and imperfect information games, as well as traditional multiagent environments such as (partially- and fully- observable) grid worlds and social dilemmas. OpenSpiel also includes tools to analyze learning dynamics and other common evaluation metrics. This document serves both as an overview of the code base and an introduction to the terminology, core concepts, and algorithms across the fields of reinforcement learning, computational game theory, and search.

Key facts. Cited by 180 papers in the map (data through 7 Oct 2026). First posted on arXiv 26 Aug 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 30; 2022: 31; 2023: 24; 2024: 27; 2025: 26; 2026: 42.

180 papers.

Sort:

By area:

  1. 2022Mastering the Game of Stratego with Model-Free Multiagent Reinforcement Learningcited by 129
  2. 2022SMACv2: An Improved Benchmark for Cooperative Multi-Agent Reinforcement Learningcited by 118
  3. 2024GTBench: Uncovering the Strategic Reasoning Limitations of LLMs via Game-Theoretic Evaluationscited by 118
  4. 2021Scalable Evaluation of Multi-Agent Reinforcement Learning with Melting Potcited by 97
  5. 2021DouZero: Mastering DouDizhu with Self-Play Deep Reinforcement Learningcited by 65
  6. 2022A Unified Approach to Reinforcement Learning, Quantal Response Equilibria, and Two-Player Zero-Sum Gamescited by 64
  7. 2021Scaling Scaling Laws with Board Gamescited by 62
  8. 2023TorchRL: A data-driven decision-making library for PyTorchcited by 59
  9. 2021Muesli: Combining Improvements in Policy Optimizationcited by 53
  10. 2023Jumanji: a Diverse Suite of Scalable Reinforcement Learning Environments in JAXcited by 52
  11. 2023Pgx: Hardware-Accelerated Parallel Game Simulators for Reinforcement Learningcited by 45
  12. 2025A Survey on the Optimization of Large Language Model-based Agentscited by 45
38 more
  1. 2022Scaling Laws for a Multi-Agent Reinforcement Learning Modelcited by 37
  2. 2021XDO: A Double Oracle Algorithm for Extensive-Form Gamescited by 34
  3. 2021ABIDES-Gym: Gym Environments for Multi-Agent Discrete Event Simulation and Application to Financial Marketscited by 34
  4. 2022Scalable Deep Reinforcement Learning Algorithms for Mean Field Gamescited by 33
  5. 2024Steering Language Models with Game-Theoretic Solverscited by 28
  6. 2025Code World Models for General Game Playingcited by 25
  7. 2022ESCHER: Eschewing Importance Sampling in Games by Computing a History Value Function to Estimate Regretcited by 24
  8. 2024Sample-Efficient Robust Multi-Agent Reinforcement Learning in the Face of Environmental Uncertaintycited by 24
  9. 2021MALib: A Parallel Framework for Population-based Multi-agent Reinforcement Learningcited by 23
  10. 2021Student of Games: A unified learning algorithm for both perfect and imperfect information gamescited by 22
  11. 2021Multi-Agent Training beyond Zero-Sum with Correlated Equilibrium Meta-Solverscited by 21
  12. 2021Exploration-Exploitation in Multi-Agent Competition: Convergence with Bounded Rationalitycited by 20
  13. 2024AdaSociety: An Adaptive Environment with Social Structures for Multi-Agent Decision-Makingcited by 19
  14. 2021Online Double Oraclecited by 18
  15. 2022Regularized Gradient Descent Ascent for Two-Player Zero-Sum Markov Gamescited by 18
  16. 2023Evolving Diverse Red-team Language Models in Multi-round Multi-agent Gamescited by 17
  17. 2024Natural Language Reinforcement Learningcited by 17
  18. 2025GEM: A Gym for Agentic LLMscited by 17
  19. 2021Solving Common-Payoff Games with Approximate Policy Iterationcited by 15
  20. 2021Efficient Deviation Types and Learning for Hindsight Rationality in Extensive-Form Gamescited by 14
  21. 2023Games for Artificial Intelligence Research: A Review and Perspectivescited by 14
  22. 2023MiniZero: Comparative Analysis of AlphaZero and MuZero on Go, Othello, and Atari Gamescited by 14
  23. 2025MARSHAL: Incentivizing Multi-Agent Reasoning via Self-Play with Strategic LLMscited by 14
  24. 2025Reevaluating Policy Gradient Methods for Imperfect-Information Gamescited by 13
  25. 2021Model-Free Online Learning in Unknown Sequential Decision Making Problems and Gamescited by 12
  26. 2022A Survey on Reinforcement Learning Methods in Character Animationcited by 12
  27. 2024PUZZLES: A Benchmark for Neural Algorithmic Reasoningcited by 12
  28. 2022Self-Play PSRO: Toward Optimal Populations in Two-Player Zero-Sum Gamescited by 11
  29. 2022Adapting to game trees in zero-sum imperfect information gamescited by 11
  30. 2025Evaluating Generalization Capabilities of LLM-Based Agents in Mixed-Motive Scenarios Using Concordiacited by 11
  31. 2021Measuring the Non-Transitivity in Chesscited by 10
  32. 2023Provably Safe Reinforcement Learning with Step-wise Violation Constraintscited by 10
  33. 2023Evaluating Agents using Social Choice Theorycited by 10
  34. 2024Policy Space Response Oracles: A Surveycited by 10
  35. 2026Verifiable Process Rewards for Agentic Reasoningcited by 10
  36. 2022Anytime PSRO for Two-Player Zero-Sum Gamescited by 9
  37. 2023Can We Find Nash Equilibria at a Linear Rate in Markov Games?cited by 9
  38. 2022Actual Causality and Responsibility Attribution in Decentralized Partially Observable Markov Decision Processescited by 8

50 most cited of 180 All 180 (JSON)