mlsanityData to 7 Oct 2026

Policy Diagnosis via Measuring Role Diversity in Cooperative Multi-agent RL

arXiv 2207.05683 ↗First posted 1 Jun 2022cs.MA cs.AI cs.LGcited by 11

Not on the map. Why?

Abstract (arXiv)

Cooperative multi-agent reinforcement learning (MARL) is making rapid progress for solving tasks in a grid world and real-world scenarios, in which agents are given different attributes and goals, resulting in different behavior through the whole multi-agent task. In this study, we quantify the agent's behavior difference and build its relationship with the policy performance via bf Role Diversity, a metric to measure the characteristics of MARL tasks. We define role diversity from three perspectives: action-based, trajectory-based, and contribution-based to fully measure a multi-agent task. Through theoretical analysis, we find that the error bound in MARL can be decomposed into three parts that have a strong relation to the role diversity. The decomposed factors can significantly impact policy optimization on three popular directions including parameter sharing, communication mechanism, and credit assignment. The main experimental platforms are based on bf Multiagent Particle Environment (MPE) and bf The StarCraft Multi-Agent Challenge (SMAC). Extensive experiments clearly show that role diversity can serve as a robust measurement for the characteristics of a multi-agent cooperation task and help diagnose whether the policy fits the current multi-agent system for a better policy performance.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 40 of its 66 references that are in the map. First posted on arXiv 1 Jun 2022.

Builds on

40 of 66 references.

References · 40

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  3. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,829
  4. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  5. 2015Trust Region Policy Optimizationcited by 2,503
  6. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
  7. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,519
  8. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,084
  9. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,013
  10. 2017Rainbow: Combining Improvements in Deep Reinforcement Learningcited by 774
  11. 2017Counterfactual Multi-Agent Policy Gradientscited by 733
  12. 2017Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithmcited by 723
28 more
  1. 2018QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 697
  2. 2019The StarCraft Multi-Agent Challengecited by 589
  3. 2017Nonparametric regression using deep neural networks with ReLU activation functioncited by 431
  4. 2016Learning Multiagent Communication with Backpropagationcited by 355
  5. 2019Emergent Tool Use From Multi-Agent Autocurriculacited by 308
  6. 2016Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivationcited by 282
  7. 2019Google Research Football: A Novel Reinforcement Learning Environmentcited by 257
  8. 2019A Theoretical Analysis of Deep Q-Learningcited by 212
  9. 2020Benchmarking Multi-Agent Deep Reinforcement Learning Algorithms in Cooperative Taskscited by 207
  10. 2018Mean Field Multi-Agent Reinforcement Learningcited by 202
  11. 2019The Hanabi Challenge: A New Frontier for AI Researchcited by 194
  12. 2018Learning Attentional Communication for Multi-Agent Cooperationcited by 170
  13. 2020Emergent Multi-Agent Communication in the Deep Learning Eracited by 168
  14. 2019Mastering Complex Control in MOBA Games with Deep Reinforcement Learningcited by 150
  15. 2020ROMA: Multi-Agent Reinforcement Learning with Emergent Rolescited by 135
  16. 2020RODE: Learning Roles to Decompose Multi-Agent Taskscited by 125
  17. 2020SMARTS: Scalable Multi-Agent Reinforcement Learning Training School for Autonomous Drivingcited by 121
  18. 2018Learning when to Communicate at Scale in Multiagent Cooperative and Competitive Taskscited by 114
  19. 2019Learning to Schedule Communication in Multi-agent Reinforcement Learningcited by 89
  20. 2021Scaling Multi-Agent Reinforcement Learning with Selective Parameter Sharingcited by 85
  21. 2017Coordinated Multi-Agent Imitation Learningcited by 70
  22. 2017MAgent: A Many-Agent Reinforcement Learning Platform for Artificial Collective Intelligencecited by 69
  23. 2021DouZero: Mastering DouDizhu with Self-Play Deep Reinforcement Learningcited by 65
  24. 2020Suphx: Mastering Mahjong with Deep Reinforcement Learningcited by 55
  25. 2021Towards Distraction-Robust Active Visual Trackingcited by 30
  26. 2020Game Plan: What AI can do for Football, and What Football can do for AIcited by 27
  27. 2021Learning in Nonzero-Sum Stochastic Games with Potentialscited by 25
  28. 2021The Neural MMO Platform for Massively Multiagent Researchcited by 17

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 4; 2024: 4; 2025: 1; 2026: 1.

11 papers.

  1. 2022Contrastive Identity-Aware Learning for Multi-Agent Value Decompositioncited by 19
  2. 2023System Neural Diversity: Measuring Behavioral Heterogeneity in Multi-Agent Learningcited by 12
  3. 2024Controlling Behavioral Diversity in Multi-Agent Reinforcement Learningcited by 10
  4. 2024Heterogeneous Multi-Agent Reinforcement Learning for Zero-Shot Scalable Collaborationcited by 6
  5. 2024Measuring Policy Distance for Multi-Agent Reinforcement Learningcited by 4
  6. 2024Novelty-Guided Data Reuse for Efficient and Diversified Multi-Agent Reinforcement Learningcited by 4
  7. 2023Iteratively Learn Diverse Strategies with State Distance Informationcited by 3
  8. 2023Presenting Multiagent Challenges in Team Sports Analyticscited by 2
  9. 2023On Diagnostics for Understanding Agent Training Behaviour in Cooperative MARLcited by 2
  10. 2025LOD: Latent Objective Discovery in Heterogeneous Multi-Agent Reinforcement Learningcited by 0
  11. 2026Graph-SND: Sparse Aggregation for Behavioral Diversity in Multi-Agent Reinforcement Learningcited by 0