mlsanityData to 14 Oct 2026

Heterogeneous Multi-agent Zero-Shot Coordination by Coevolution

arXiv 2208.04957 ↗First posted 9 Aug 2022cs.NE cs.AI cs.LGcited by 12

Not on the map. Why?

Abstract (arXiv)

Generating agents that can achieve zero-shot coordination (ZSC) with unseen partners is a new challenge in cooperative multi-agent reinforcement learning (MARL). Recently, some studies have made progress in ZSC by exposing the agents to diverse partners during the training process. They usually involve self-play when training the partners, implicitly assuming that the tasks are homogeneous. However, many real-world tasks are heterogeneous, and hence previous methods may be inefficient. In this paper, we study the heterogeneous ZSC problem for the first time and propose a general method based on coevolution, which coevolves two populations of agents and partners through three sub-processes: pairing, updating and selection. Experimental results on various heterogeneous tasks highlight the necessity of considering the heterogeneous setting and demonstrate that our proposed method is a promising solution for heterogeneous ZSC tasks.

Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Builds on 34 of its 70 references that are in the map. First posted on arXiv 9 Aug 2022.

Builds on

34 of 70 references.

References · 34

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  3. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,103
  4. 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 752
  5. 2017Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithmcited by 734
  6. 2020Curriculum Learning for Reinforcement Learning Domains: A Framework and Surveycited by 373
  7. 2017Population Based Training of Neural Networkscited by 367
  8. 2019On the Utility of Learning about Humans for Human-AI Coordinationcited by 349
  9. 2017Deep Reinforcement Learning: An Overviewcited by 333
  10. 2017StarCraft II: A New Challenge for Reinforcement Learningcited by 274
  11. 2014Robots that can adapt like animalscited by 204
  12. 2020"Other-Play" for Zero-Shot Coordinationcited by 198
22 more
  1. 2021Open-Ended Learning Leads to Generally Capable Agentscited by 182
  2. 2022Evolving Curricula with Regret-Based Environment Designcited by 173
  3. 2020Automatic Curriculum Learning For Deep RL: A Short Surveycited by 150
  4. 2021Collaborating with Humans without Human Datacited by 142
  5. 2020Effective Diversity in Population Based Reinforcement Learningcited by 119
  6. 2017Quality and Diversity Optimization: A Unifying Modular Frameworkcited by 113
  7. 2017Improving Exploration in Evolution Strategies for Deep Reinforcement Learning via a Population of Novelty-Seeking Agentscited by 107
  8. 2020Enhanced POET: Open-Ended Reinforcement Learning through Unbounded Invention of Learning Challenges and their Solutionscited by 106
  9. 2020One Solution is Not All You Need: Few-Shot Extrapolation via Structured MaxEnt RLcited by 79
  10. 2021Off-Belief Learningcited by 75
  11. 2021Maximum Entropy Population-Based Training for Zero-Shot Human-AI Coordinationcited by 70
  12. 2020Quality-Diversity Optimization: a novel branch of stochastic optimizationcited by 60
  13. 2020Illuminating Mario Scenes in the Latent Space of a Generative Adversarial Networkcited by 52
  14. 2022A Survey of Ad Hoc Teamwork Researchcited by 43
  15. 2021On the Approximation of Cooperative Heterogeneous Multi-Agent Reinforcement Learning (MARL) using Mean Field Control (MFC)cited by 32
  16. 2022Any-Play: An Intrinsic Augmentation for Zero-Shot Coordinationcited by 23
  17. 2020Ridge Rider: Finding Diverse Solutions by Following Eigenvectors of the Hessiancited by 20
  18. 2021Mis-spoke or mis-lead: Achieving Robustness in Multi-Agent Communicative Reinforcement Learningcited by 18
  19. 2021Co-GAIL: Learning Diverse Strategies for Human-Robot Collaborationcited by 17
  20. 2020Heterogeneous Multi-Agent Reinforcement Learning for Unknown Environment Mappingcited by 13
  21. 2022Heterogeneous-Agent Mirror Learning: A Continuum of Solutions to Cooperative MARLcited by 12
  22. 2021Few-shot Quality-Diversity Optimizationcited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 7; 2024: 1; 2025: 3; 2026: 1.

12 papers.

  1. 2023Cooperative Open-ended Learning Framework for Zero-shot Coordinationcited by 30
  2. 2023ZSC-Eval: An Evaluation Toolkit and Benchmark for Multi-agent Zero-shot Coordinationcited by 24
  3. 2023Robust multi-agent coordination via evolutionary generation of auxiliary adversarial attackerscited by 15
  4. 2023Tackling Cooperative Incompatibility for Zero-Shot Human-AI Coordinationcited by 12
  5. 2023Multi-agent Continual Coordination via Progressive Task Contextualizationcited by 11
  6. 2023Efficient Human-AI Coordination via Preparatory Language-based Conventioncited by 7
  7. 2023Communication-Robust Multi-Agent Learning by Adaptable Auxiliary Multi-Agent Adversary Generationcited by 6
  8. 2025The Yokai Learning Environment: Tracking Beliefs Over Space and Timecited by 3
  9. 2024HOLA-Drone: Hypergraphic Open-ended Learning for Zero-Shot Multi-Drone Cooperative Pursuitcited by 1
  10. 2025Generalizable Agent Modeling for Agent Collaboration-Competition Adaptation with Multi-Retrieval and Dynamic Generationcited by 0
  11. 2025CooT: Learning to Coordinate In-Context with Coordination Transformerscited by 0
  12. 2026Zero Shot Coordination for Sparse Reward Tasks with Diverse Reward Shapingscited by 0