mlsanityData to 14 Oct 2026

Policy Space Response Oracles: A Survey

arXiv 2403.02227 ↗First posted 4 Mar 2024cs.GT cs.AI cs.MAcited by 10

Not on the map. Why?

Abstract (arXiv)

Game theory provides a mathematical way to study the interaction between multiple decision makers. However, classical game-theoretic analysis is limited in scalability due to the large number of strategies, precluding direct application to more complex scenarios. This survey provides a comprehensive overview of a framework for large games, known as Policy Space Response Oracles (PSRO), which holds promise to improve scalability by focusing attention on sufficient subsets of strategies. We first motivate PSRO and provide historical context. We then focus on the strategy exploration problem for PSRO: the challenge of assembling effective subsets of strategies that still represent the original game well with minimum computational cost. We survey current research directions for enhancing the efficiency of PSRO, and explore the applications of PSRO across various domains. We conclude by discussing open questions and future research.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Builds on 53 of its 109 references that are in the map. First posted on arXiv 4 Mar 2024.

Builds on

53 of 109 references.

References · 53

  1. 2017A Unified Game-Theoretic Approach to Multiagent Reinforcement Learningcited by 268
  2. 2019OpenSpiel: A Framework for Reinforcement Learning in Gamescited by 186
  3. 2022Mastering the Game of Stratego with Model-Free Multiagent Reinforcement Learningcited by 131
  4. 2018Deep Counterfactual Regret Minimizationcited by 103
  5. 2019Open-ended Learning in Symmetric Zero-sum Gamescited by 96
  6. 2020Game-Theoretic Multiagent Reinforcement Learningcited by 94
  7. 2020Real World Games Look Like Spinning Topscited by 66
  8. 2019A Generalized Training Approach for Multiagent Learningcited by 54
  9. 2019α-Rank: Multi-Agent Evaluation by Evolutioncited by 51
  10. 2018Re-evaluating Evaluationcited by 47
  11. 2020Pipeline PSRO: A Scalable Approach for Finding Approximate Nash Equilibria in Large Gamescited by 47
  12. 2021Modelling Behavioural Diversity for Learning in Open-Ended Gamescited by 39
38 more
  1. 2021XDO: A Double Oracle Algorithm for Extensive-Form Gamescited by 34
  2. 2024Steering Language Models with Game-Theoretic Solverscited by 28
  3. 2018Depth-Limited Solving for Imperfect-Information Gamescited by 27
  4. 2023ASP: Learn a Universal Neural Solver!cited by 27
  5. 2021Diverse Auto-Curriculum is Critical for Successful Real-World Multiagent Learning Systemscited by 24
  6. 2022ESCHER: Eschewing Importance Sampling in Games by Computing a History Value Function to Estimate Regretcited by 24
  7. 2021MALib: A Parallel Framework for Population-based Multi-agent Reinforcement Learningcited by 23
  8. 2021Multi-Agent Training beyond Zero-Sum with Correlated Equilibrium Meta-Solverscited by 21
  9. 2017Computational Results for Extensive-Form Adversarial Team Gamescited by 19
  10. 2018Deep Reinforcement Learning for Green Security Games with Real-Time Informationcited by 19
  11. 2021Online Double Oraclecited by 18
  12. 2023Policy Space Diversity for Non-Transitive Gamescited by 15
  13. 2023Diversifying AI: Towards Creative Chess with AlphaZerocited by 15
  14. 2021A Game-Theoretic Approach for Improving Generalization Ability of TSP Solverscited by 14
  15. 2021Learning Equilibria in Mean-Field Games: Introducing Mean-Field PSROcited by 14
  16. 2022A Game-Theoretic Framework for Managing Risk in Multi-Agent Systemscited by 14
  17. 2019Finding Needles in a Moving Haystack: Prioritizing Alerts with Adversarial Reinforcement Learningcited by 12
  18. 2021Robust Reinforcement Learning Under Minimax Regret for Green Securitycited by 12
  19. 2021Sample-based Approximation of Nash in Large Many-Player Games via Gradient Descentcited by 11
  20. 2022Self-Play PSRO: Toward Optimal Populations in Two-Player Zero-Sum Gamescited by 11
  21. 2019αα-Rank: Practically Scaling α-Rank through Stochastic Optimisationcited by 10
  22. 2021Measuring the Non-Transitivity in Chesscited by 10
  23. 2022Efficient Policy Space Response Oraclescited by 8
  24. 2022Simplex Neural Population Learning: Any-Mixture Bayes-Optimality in Symmetric Zero-sum Gamescited by 8
  25. 2021Evaluating Strategy Exploration in Empirical Game-Theoretic Analysiscited by 7
  26. 2021DO-GAN: A Double Oracle Framework for Generative Adversarial Networkscited by 6
  27. 2023Combining Tree-Search, Generative Models, and Nash Bargaining Concepts in Game-Theoretic Reinforcement Learningcited by 6
  28. 2023Computing Optimal Equilibria and Mechanisms via Learning in Zero-Sum Extensive-Form Gamescited by 6
  29. 2023An Empirical Study on Google Research Football Multi-agent Scenarioscited by 5
  30. 2018Beyond Local Nash Equilibria for Adversarial Networkscited by 4
  31. 2023Regret-Minimizing Double Oracle for Extensive-Form Gamescited by 4
  32. 2023Co-Learning Empirical Games and World Modelscited by 4
  33. 2023Double Oracle Algorithm for Game-Theoretic Robot Allocation on Graphscited by 4
  34. 2024Neural Population Learning beyond Symmetric Zero-sum Gamescited by 4
  35. 2021Empirical Game-Theoretic Analysis for Mean Field Gamescited by 3
  36. 2019Competitive Influence Maximization: Integrating Budget Allocation and Seed Selectioncited by 2
  37. 2023Exploiting Extensive-Form Structure in Empirical Game-Theoretic Analysiscited by 2
  38. 2023Regularization for Strategy Exploration in Empirical Game-Theoretic Analysiscited by 2

50 most cited of 53 All 53 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 5; 2026: 4.

10 papers.

  1. 2024A Survey on Self-play Methods in Reinforcement Learningcited by 26
  2. 2025Faster Rates for No-Regret Learning in General Games via Cautious Optimismcited by 10
  3. 2026Discovering Multiagent Learning Algorithms with Large Language Modelscited by 4
  4. 2025NashPG: A Policy Gradient Method with Iteratively Refined Regularization for Finding Nash Equilibriacited by 2
  5. 2026Code-Space Response Oracles: Generating Interpretable Multi-Agent Policies with Large Language Modelscited by 2
  6. 2026Conservative Equilibrium Discovery in Offline Game-Theoretic Multiagent Reinforcement Learningcited by 1
  7. 2025PoolFlip: A Multi-Agent Reinforcement Learning Security Environment for Cyber Defensecited by 0
  8. 2025The Hidden Game Problemcited by 0
  9. 2025Perturbing Best Responses in Zero-Sum Gamescited by 0
  10. 2026IB-RL: Isolated Bilateral Reinforcement Learning for Strategic Dialogue Agentscited by 0