mlsanityData to 7 Oct 2026

"Other-Play" for Zero-Shot Coordination

arXiv 2003.02979 ↗First posted 6 Mar 2020cs.AIcited by 194

Not on the map. Why?

Abstract (arXiv)

We consider the problem of zero-shot coordination - constructing AI agents that can coordinate with novel partners they have not seen before (e.g. humans). Standard Multi-Agent Reinforcement Learning (MARL) methods typically focus on the self-play (SP) setting where agents construct strategies by playing the game with themselves repeatedly. Unfortunately, applying SP naively to the zero-shot coordination problem can produce agents that establish highly specialized conventions that do not carry over to novel partners they have not been trained with. We introduce a novel learning algorithm called other-play (OP), that enhances self-play by looking for more robust strategies, exploiting the presence of known symmetries in the underlying problem. We characterize OP theoretically as well as experimentally. We study the cooperative card game Hanabi and show that OP agents achieve higher scores when paired with independently trained agents. In preliminary results we also show that our OP agents obtains higher average scores when paired with human players, compared to state-of-the-art SP agents.

Key facts. Cited by 194 papers in the map (data through 7 Oct 2026). First posted on arXiv 6 Mar 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 29; 2022: 31; 2023: 37; 2024: 21; 2025: 47; 2026: 29.

194 papers.

Sort:

By area:

  1. 2023SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agentscited by 350
  2. 2023Habitat 3.0: A Co-Habitat for Humans, Avatars and Robotscited by 220
  3. 2021A Survey of Zero-shot Generalisation in Deep Reinforcement Learningcited by 164
  4. 2021Collaborating with Humans without Human Datacited by 139
  5. 2023Human-Timescale Adaptation in an Open-Ended Task Spacecited by 136
  6. 2021Scalable Evaluation of Multi-Agent Reinforcement Learning with Melting Potcited by 97
  7. 2021Off-Belief Learningcited by 72
  8. 2021Maximum Entropy Population-Based Training for Zero-Shot Human-AI Coordinationcited by 69
  9. 2023LLM-Coordination: Evaluating and Analyzing Multi-agent Coordination Abilities in Large Language Modelscited by 65
  10. 2023Language Instructed Reinforcement Learning for Human-AI Coordinationcited by 61
  11. 2023JaxMARL: Multi-Agent RL Environments and Algorithms in JAXcited by 59
  12. 2022Nocturne: a scalable driving benchmark for bringing multi-agent learning one step closer to the real worldcited by 56
38 more
  1. 2021Modeling Strong and Human-Like Gameplay with KL-Regularized Searchcited by 50
  2. 2023LLM-Powered Hierarchical Language Agent for Real-time Human-AI Coordinationcited by 48
  3. 2022Mastering the Game of No-Press Diplomacy via Human-Regularized Reinforcement Learning and Planningcited by 47
  4. 2022A Survey of Ad Hoc Teamwork Researchcited by 41
  5. 2025Infrastructure for AI Agentscited by 40
  6. 2021Evaluation of Human-AI Teams for Learned and Rule-Based Agents in Hanabicited by 37
  7. 2023Learning Zero-Shot Cooperation with Humans, Assuming Humans Are Biasedcited by 37
  8. 2025Vision-Zero: Scalable VLM Self-Improvement via Strategic Gamified Self-Playcited by 37
  9. 2021Discovering Diverse Multi-Agent Strategic Behavior via Reward Randomizationcited by 35
  10. 2021Few-shot Language Coordination by Modeling Theory of Mindcited by 32
  11. 2022Revisiting Some Common Practices in Cooperative Multi-Agent Reinforcement Learningcited by 32
  12. 2021Continuous Coordination As a Realistic Scenario for Lifelong Learningcited by 31
  13. 2021A New Formalism, Method and Open Issues for Zero-Shot Coordinationcited by 31
  14. 2022The Boltzmann Policy Distribution: Accounting for Systematic Suboptimality in Human Modelscited by 31
  15. 2022K-level Reasoning for Zero-Shot Coordination in Hanabicited by 30
  16. 2023Cooperative Open-ended Learning Framework for Zero-shot Coordinationcited by 30
  17. 2023A Review of Cooperation in Multi-agent Learningcited by 27
  18. 2021On the Critical Role of Conventions in Adaptive Human-AI Collaborationcited by 26
  19. 2021Evaluating the Robustness of Collaborative Agentscited by 25
  20. 2023PECAN: Leveraging Policy Ensemble for Context-Aware Zero-Shot Human-AI Coordinationcited by 25
  21. 2021Multi-Agent MDP Homomorphic Networkscited by 24
  22. 2022Law Informs Code: A Legal Informatics Approach to Aligning Artificial Intelligence with Humanscited by 23
  23. 2023NOPA: Neurally-guided Online Probabilistic Assistance for Building Socially Intelligent Home Assistantscited by 23
  24. 2023ZSC-Eval: An Evaluation Toolkit and Benchmark for Multi-agent Zero-shot Coordinationcited by 23
  25. 2024Learning to Cooperate with Humans using Generative Agentscited by 23
  26. 2022Any-Play: An Intrinsic Augmentation for Zero-Shot Coordinationcited by 22
  27. 2021Quantifying the effects of environment and population diversity in multi-agent reinforcement learningcited by 21
  28. 2021Emergent Discrete Communication in Semantic Spacescited by 20
  29. 2021Quasi-Equivalence Discovery for Zero-Shot Emergent Communicationcited by 19
  30. 2021Scalable Online Planning via Reinforcement Learning Fine-Tuningcited by 18
  31. 2023A Model-Based Solution to the Offline Multi-Agent Reinforcement Learning Coordination Problemcited by 18
  32. 2021Emergent Communication under Competitioncited by 17
  33. 2022Generating Teammates for Training Robust Ad Hoc Teamwork Agents via Best-Response Diversitycited by 17
  34. 2023Asynchronous Multi-Agent Reinforcement Learning for Efficient Real-Time Multi-Robot Cooperative Explorationcited by 17
  35. 2024Human-compatible driving partners through data-regularized self-play reinforcement learningcited by 17
  36. 2022Grounding Aleatoric Uncertainty for Unsupervised Environment Designcited by 16
  37. 2022Equivariant Networks for Zero-Shot Coordinationcited by 16
  38. 2024Refining Minimax Regret for Unsupervised Environment Designcited by 16

50 most cited of 194 All 194 (JSON)