mlsanityData to 13 Oct 2026

CORD: Generalizable Cooperation via Role Diversity

arXiv 2501.02221 ↗First posted 4 Jan 2025cs.AI cs.LG cs.MAcited by 1

On the map: Deep reinforcement learning methods → RL for games and combinatorial optimization

Abstract (arXiv)

Cooperative multi-agent reinforcement learning (MARL) aims to develop agents that can collaborate effectively. However, most cooperative MARL methods overfit training agents, making learned policies not generalize well to unseen collaborators, which is a critical issue for real-world deployment. Some methods attempt to address the generalization problem but require prior knowledge or predefined policies of new teammates, limiting real-world applications. To this end, we propose a hierarchical MARL approach to enable generalizable cooperation via role diversity, namely CORD. CORD's high-level controller assigns roles to low-level agents by maximizing the role entropy with constraints. We show this constrained objective can be decomposed into causal influence in role that enables reasonable role assignment, and role heterogeneity that yields coherent, non-redundant role clusters. Evaluated on a variety of cooperative multi-agent tasks, CORD achieves better performance than baselines, especially in generalization tests. Ablation studies further demonstrate the efficacy of the constrained objective in generalizable cooperation.

Key facts. Cited by 1 paper in the map (data through 13 Oct 2026). Placed in RL for games and combinatorial optimization, one of 15 directions in Deep reinforcement learning methods. Builds on 34 of its 61 references that are in the map. First posted on arXiv 4 Jan 2025.

Builds on

34 of 61 references.

References · 34

  1. 2017Attention Is All You Needcited by 60,106
  2. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,539
  3. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,525
  4. 2021The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Gamescited by 910
  5. 2019The StarCraft Multi-Agent Challengecited by 592
  6. 2020Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 478
  7. 2016The Option-Critic Architecturecited by 439
  8. 2019QTRAN: Learning to Factorize with Transformation for Cooperative Multi-Agent Reinforcement Learningcited by 374
  9. 2019Dynamics-Aware Unsupervised Discovery of Skillscited by 309
  10. 2020QPLEX: Duplex Dueling Multi-Agent Q-Learningcited by 268
  11. 2016Variational Intrinsic Controlcited by 264
  12. 2024Reinforcement learningcited by 238
22 more
  1. 2020"Other-Play" for Zero-Shot Coordinationcited by 194
  2. 2021Trust Region Policy Optimisation in Multi-Agent Reinforcement Learningcited by 182
  3. 2019Planning with Goal-Conditioned Policiescited by 145
  4. 2020ROMA: Multi-Agent Reinforcement Learning with Emergent Rolescited by 135
  5. 2020RODE: Learning Roles to Decompose Multi-Agent Taskscited by 125
  6. 2020Effective Diversity in Population Based Reinforcement Learningcited by 116
  7. 2018Near-Optimal Representation Learning for Hierarchical Reinforcement Learningcited by 108
  8. 2020Learning Robot Skills with Temporal Variational Inferencecited by 54
  9. 2021Multi-Agent Reinforcement Learning for Active Voltage Control on Power Distribution Networkscited by 52
  10. 2022Interventions, Where and How? Experimental Design for Causal Models at Scalecited by 49
  11. 2020Randomized Entity-wise Factorization for Multi-Agent Reinforcement Learningcited by 38
  12. 2021Coach-Player Multi-Agent Reinforcement Learning for Dynamic Team Compositioncited by 29
  13. 2018Learning Goal Embeddings via Self-Play for Hierarchical Reinforcement Learningcited by 23
  14. 2019A Cooperative Multi-Agent Reinforcement Learning Framework for Resource Balancing in Complex Logistics Networkcited by 22
  15. 2022Generalization in Cooperative Multi-Agent Systemscited by 19
  16. 2022More Centralized Training, Still Decentralized Execution: Multi-Agent Conditional Policy Factorizationcited by 19
  17. 2020Multi-Agent Collaboration via Reward Attribution Decompositioncited by 17
  18. 2022ALMA: Hierarchical Learning for Composite Multi-Agent Taskscited by 17
  19. 2019Hierarchical Policy Learning is Sensitive to Goal Space Designcited by 8
  20. 2022Decentralized Policy Optimizationcited by 5
  21. 2022Multi-Agent Reinforcement Learning with Shared Resources for Inventory Managementcited by 5
  22. 2023Improving Zero-Shot Coordination Performance Based on Policy Similaritycited by 1

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025Automata-Conditioned Cooperative Multi-Agent Reinforcement Learningcited by 1