mlsanityData to 14 Oct 2026

Efficient Reinforcement Learning for Zero-Shot Coordination in Evolving Games

arXiv 2511.11083 ↗First posted 14 Nov 2025cs.LG cs.AIcited by 0

On the map: Deep reinforcement learning methods → RL for games and combinatorial optimization

Abstract (arXiv)

Zero-shot coordination(ZSC), a key challenge in multi-agent game theory, has become a hot topic in reinforcement learning (RL) research recently, especially in complex evolving games. It focuses on the generalization ability of agents, requiring them to coordinate well with collaborators from a diverse, potentially evolving, pool of partners that are not seen before without any fine-tuning. Population-based training, which approximates such an evolving partner pool, has been proven to provide good zero-shot coordination performance; nevertheless, existing methods are limited by computational resources, mainly focusing on optimizing diversity in small populations while neglecting the potential performance gains from scaling population size. To address this issue, this paper proposes the Scalable Population Training (ScaPT), an efficient RL training framework comprising two key components: a meta-agent that efficiently realizes a population by selectively sharing parameters across agents, and a mutual information regularizer that guarantees population diversity. To empirically validate the effectiveness of ScaPT, this paper evaluates it along with representational frameworks in Hanabi cooperative game and confirms its superiority.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for games and combinatorial optimization, one of 15 directions in Deep reinforcement learning methods. Builds on 22 of its 31 references that are in the map. First posted on arXiv 14 Nov 2025.

Builds on

22 of 31 references.

References · 22

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,745
  3. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,558
  4. 2015Prioritized Experience Replaycited by 1,099
  5. 2021The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Gamescited by 942
  6. 2015Dueling Network Architectures for Deep Reinforcement Learningcited by 785
  7. 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 752
  8. 2017Population Based Training of Neural Networkscited by 367
  9. 2019Dynamics-Aware Unsupervised Discovery of Skillscited by 313
  10. 2018Learning to Adapt in Dynamic, Real-World Environments Through Meta-Reinforcement Learningcited by 276
  11. 2020"Other-Play" for Zero-Shot Coordinationcited by 198
  12. 2018Meta-Reinforcement Learning of Structured Exploration Strategiescited by 130
10 more
  1. 2021Off-Belief Learningcited by 75
  2. 2021Maximum Entropy Population-Based Training for Zero-Shot Human-AI Coordinationcited by 70
  3. 2019Simplified Action Decoder for Deep Multi-Agent Reinforcement Learningcited by 48
  4. 2021A New Formalism, Method and Open Issues for Zero-Shot Coordinationcited by 33
  5. 2022K-level Reasoning for Zero-Shot Coordination in Hanabicited by 32
  6. 2022Any-Play: An Intrinsic Augmentation for Zero-Shot Coordinationcited by 23
  7. 2022Generating Teammates for Training Robust Ad Hoc Teamwork Agents via Best-Response Diversitycited by 17
  8. 2022Equivariant Networks for Zero-Shot Coordinationcited by 16
  9. 2025SRMT: Shared Memory for Multi-agent Lifelong Pathfindingcited by 8
  10. 2023Towards Effective and Interpretable Human-Agent Collaboration in MOBA Games: A Communication Perspectivecited by 5

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Is Inter-Seed Cross-Play Enough? Evaluating the Robustness of Zero-Shot Coordination Algorithms to Implementation Details
  2. 2025Cross-environment Cooperation Enables Zero-shot Multi-agent Coordination
  3. 2025Efficient Generation of Diverse Cooperative Agents with World Models
  4. 2026Zero Shot Coordination for Sparse Reward Tasks with Diverse Reward Shapings
  5. 2024Noisy Zero-Shot Coordination: Breaking The Common Knowledge Assumption In Zero-Shot Coordination Games
  6. 2026Theory of Mind Guided Strategy Adaptation for Zero-Shot Coordination
  7. 2026PopGen: Generating Semantically Diverse Populations for Zero-Shot Coordination
  8. 2024KnowPC: Knowledge-Driven Programmatic Reinforcement Learning for Zero-shot Coordination