mlsanityData to 7 Oct 2026

A Unified Game-Theoretic Approach to Multiagent Reinforcement Learning

arXiv 1711.00832 ↗First posted 2 Nov 2017cs.AI cs.GT cs.LGcited by 265

Not on the map. Why?

Abstract (arXiv)

To achieve general intelligence, agents must learn how to interact with others in a shared environment: this is the challenge of multiagent reinforcement learning (MARL). The simplest form is independent reinforcement learning (InRL), where each agent treats its experience as part of its (non-stationary) environment. In this paper, we first observe that policies learned using InRL can overfit to the other agents' policies during training, failing to sufficiently generalize during execution. We introduce a new metric, joint-policy correlation, to quantify this effect. We describe an algorithm for general MARL, based on approximate best responses to mixtures of policies generated using deep reinforcement learning, and empirical game-theoretic analysis to compute meta-strategies for policy selection. The algorithm generalizes previous ones such as InRL, iterated best response, double oracle, and fictitious play. Then, we present a scalable implementation which reduces the memory requirement using decoupled meta-solvers. Finally, we demonstrate the generality of the resulting policies in two partially observable settings: gridworld coordination games and poker.

Key facts. Cited by 265 papers in the map (data through 7 Oct 2026). First posted on arXiv 2 Nov 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2019: 2; 2020: 0; 2021: 43; 2022: 52; 2023: 45; 2024: 41; 2025: 30; 2026: 52.

265 papers.

Sort:

By area:

  1. 2024Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Modelscited by 552
  2. 2021Open-Ended Learning Leads to Generally Capable Agentscited by 177
  3. 2023Language Agents with Reinforcement Learning for Strategic Play in the Werewolf Gamecited by 148
  4. 2021Collaborating with Humans without Human Datacited by 139
  5. 2023Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learningcited by 111
  6. 2021Scalable Evaluation of Multi-Agent Reinforcement Learning with Melting Potcited by 97
  7. 2021DouZero: Mastering DouDizhu with Self-Play Deep Reinforcement Learningcited by 65
  8. 2022A Unified Approach to Reinforcement Learning, Quantal Response Equilibria, and Two-Player Zero-Sum Gamescited by 64
  9. 2024Learning Multi-Agent Communication from Graph Modeling Perspectivecited by 47
  10. 2024Evaluating Language Model Agency through Negotiationscited by 43
  11. 2024Optima: Optimizing Effectiveness and Efficiency for LLM-Based Multi-Agent Systemcited by 40
  12. 2021Modelling Behavioural Diversity for Learning in Open-Ended Gamescited by 39
38 more
  1. 2022Melting Pot 2.0cited by 37
  2. 2021Discovering Diverse Multi-Agent Strategic Behavior via Reward Randomizationcited by 35
  3. 2021XDO: A Double Oracle Algorithm for Extensive-Form Gamescited by 34
  4. 2021Independent Natural Policy Gradient Always Converges in Markov Potential Gamescited by 34
  5. 2021Learning to Simulate Self-Driven Particles System with Coordinated Policy Optimizationcited by 33
  6. 2022Scalable Deep Reinforcement Learning Algorithms for Mean Field Gamescited by 33
  7. 2025Chasing Moving Targets with Online Self-Play Reinforcement Learning for Safer Language Modelscited by 33
  8. 2021Gradient play in stochastic games: stationary points, convergence, and sample complexitycited by 32
  9. 2022Revisiting Some Common Practices in Cooperative Multi-Agent Reinforcement Learningcited by 32
  10. 2021No-Press Diplomacy from Scratchcited by 31
  11. 2022K-level Reasoning for Zero-Shot Coordination in Hanabicited by 30
  12. 2023Cooperative Open-ended Learning Framework for Zero-shot Coordinationcited by 30
  13. 2023MAESTRO: Open-Ended Environment Design for Multi-Agent Reinforcement Learningcited by 30
  14. 2023Multi-agent Reinforcement Learning: A Comprehensive Surveycited by 30
  15. 2022Adversarial Policies Beat Superhuman Go AIscited by 29
  16. 2024Steering Language Models with Game-Theoretic Solverscited by 28
  17. 2023ASP: Learn a Universal Neural Solver!cited by 27
  18. 2024A Survey on Self-play Methods in Reinforcement Learningcited by 26
  19. 2021Diverse Auto-Curriculum is Critical for Successful Real-World Multiagent Learning Systemscited by 24
  20. 2022ESCHER: Eschewing Importance Sampling in Games by Computing a History Value Function to Estimate Regretcited by 24
  21. 2021MALib: A Parallel Framework for Population-based Multi-agent Reinforcement Learningcited by 23
  22. 2022PMIC: Improving Multi-Agent Reinforcement Learning with Progressive Mutual Information Collaborationcited by 23
  23. 2022Continuously Discovering Novel Strategies via Reward-Switching Policy Optimizationcited by 23
  24. 2022On the Limitations of Elo: Real-World Games, are Transitive, not Additivecited by 23
  25. 2021Pick Your Battles: Interaction Graphs as Population-Level Objectives for Strategic Diversitycited by 22
  26. 2023Causal Reinforcement Learning: A Surveycited by 22
  27. 2021Quantifying the effects of environment and population diversity in multi-agent reinforcement learningcited by 21
  28. 2021Multi-Agent Training beyond Zero-Sum with Correlated Equilibrium Meta-Solverscited by 21
  29. 2021Deep Latent Competition: Learning to Race Using Visual Control Policies in Latent Spacecited by 20
  30. 2024A social path to human-like artificial intelligencecited by 20
  31. 2021Quasi-Equivalence Discovery for Zero-Shot Emergent Communicationcited by 19
  32. 2021Online Double Oraclecited by 18
  33. 2021Mis-spoke or mis-lead: Achieving Robustness in Multi-Agent Communicative Reinforcement Learningcited by 18
  34. 2023TiZero: Mastering Multi-Agent Football with Curriculum Learning and Self-Playcited by 18
  35. 2021Emergent Communication under Competitioncited by 17
  36. 2023Evolving Diverse Red-team Language Models in Multi-round Multi-agent Gamescited by 17
  37. 2021Co-GAIL: Learning Diverse Strategies for Human-Robot Collaborationcited by 16
  38. 2022PerfectDou: Dominating DouDizhu with Perfect Information Distillationcited by 15

50 most cited of 265 All 265 (JSON)