mlsanityData to 7 Oct 2026

Benchmarking Multi-Agent Deep Reinforcement Learning Algorithms in Cooperative Tasks

arXiv 2006.07869 ↗First posted 14 Jun 2020cs.LG cs.AI cs.MAcited by 207

Not on the map. Why?

Abstract (arXiv)

Multi-agent deep reinforcement learning (MARL) suffers from a lack of commonly-used evaluation tasks and criteria, making comparisons between approaches difficult. In this work, we provide a systematic evaluation and comparison of three different classes of MARL algorithms (independent learning, centralised multi-agent policy gradient, value decomposition) in a diverse range of cooperative multi-agent learning tasks. Our experiments serve as a reference for the expected performance of algorithms across different learning tasks, and we provide insights regarding the effectiveness of different learning approaches. We open-source EPyMARL, which extends the PyMARL codebase to include additional algorithms and allow for flexible configuration of algorithm implementation details such as parameter sharing. Finally, we open-source two environments for multi-agent research which focus on coordination under sparse rewards.

Key facts. Cited by 207 papers in the map (data through 7 Oct 2026). First posted on arXiv 14 Jun 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 6; 2022: 29; 2023: 40; 2024: 33; 2025: 63; 2026: 36.

207 papers.

Sort:

By area:

  1. 2021The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Gamescited by 903
  2. 2022A Survey of Multi-Agent Deep Reinforcement Learning with Communicationcited by 103
  3. 2023Heterogeneous-Agent Reinforcement Learningcited by 67
  4. 2023JaxMARL: Multi-Agent RL Environments and Algorithms in JAXcited by 59
  5. 2023Jumanji: a Diverse Suite of Scalable Reinforcement Learning Environments in JAXcited by 52
  6. 2022Towards a Standardised Performance Evaluation Protocol for Cooperative MARLcited by 48
  7. 2022Multi-agent Dynamic Algorithm Configurationcited by 36
  8. 2022Revisiting Some Common Practices in Cooperative Multi-Agent Reinforcement Learningcited by 32
  9. 2023BenchMARL: Benchmarking Multi-Agent Reinforcement Learningcited by 31
  10. 2022Scalable Multi-Agent Reinforcement Learning for Warehouse Logistics with Robotic and Human Co-Workerscited by 28
  11. 2023A Review of Cooperation in Multi-agent Learningcited by 27
  12. 2021SHAQ: Incorporating Shapley Value Theory into Multi-Agent Q-Learningcited by 26
38 more
  1. 2021Decoupled Reinforcement Learning to Stabilise Intrinsically-Motivated Explorationcited by 25
  2. 2025Graphs Meet AI Agents: Taxonomy, Progress, and Future Opportunitiescited by 23
  3. 2022MARLlib: A Scalable and Efficient Multi-agent Reinforcement Learning Librarycited by 21
  4. 2024POGEMA: A Benchmark Platform for Cooperative Multi-Agent Pathfindingcited by 20
  5. 2022Generating Teammates for Training Robust Ad Hoc Teamwork Agents via Best-Response Diversitycited by 17
  6. 2022An Introduction to Multi-Agent Reinforcement Learning and Review of its Application to Autonomous Mobilitycited by 16
  7. 2023Attacking Cooperative Multi-Agent Reinforcement Learning by Adversarial Minority Influencecited by 16
  8. 2024N-Agent Ad Hoc Teamworkcited by 15
  9. 2021Mava: a research library for distributed multi-agent reinforcement learning in JAXcited by 14
  10. 2022Pareto Actor-Critic for Equilibrium Selection in Multi-Agent Reinforcement Learningcited by 14
  11. 2024FightLadder: A Benchmark for Competitive Multi-Agent Reinforcement Learningcited by 13
  12. 2022Heterogeneous-Agent Mirror Learning: A Continuum of Solutions to Cooperative MARLcited by 12
  13. 2022A Survey on Large-Population Systems and Scalable Multi-Agent Reinforcement Learningcited by 12
  14. 2023Dealing With Non-stationarity in Decentralized Cooperative Multi-Agent Deep Reinforcement Learning via Multi-Timescale Learningcited by 12
  15. 2023Learning Cyber Defence Tactics from Scratch with Multi-Agent Reinforcement Learningcited by 12
  16. 2022Toward Policy Explanations for Multi-Agent Reinforcement Learningcited by 11
  17. 2022Policy Diagnosis via Measuring Role Diversity in Cooperative Multi-agent RLcited by 11
  18. 2023Multi-agent Continual Coordination via Progressive Task Contextualizationcited by 10
  19. 2023AlberDICE: Addressing Out-Of-Distribution Joint Actions in Offline Multi-Agent RL via Alternating Stationary Distribution Correction Estimationcited by 10
  20. 2024Sable: a Performant, Efficient and Scalable Sequence Model for MARLcited by 10
  21. 2023TransfQMix: Transformers for Leveraging the Graph Structure of Multi-Agent Reinforcement Learning Problemscited by 9
  22. 2023Explainable Multi-Agent Reinforcement Learning for Temporal Queriescited by 9
  23. 2023IMP-MARL: a Suite of Environments for Large-scale Infrastructure Management Planning via MARLcited by 9
  24. 2023Scalable Multi-agent Covering Option Discovery based on Kronecker Graphscited by 9
  25. 2024Dispelling the Mirage of Progress in Offline MARL through Standardised Baselines and Evaluationcited by 9
  26. 2024YOLO-MARL: You Only LLM Once for Multi-Agent Reinforcement Learningcited by 9
  27. 2021Local Patch AutoAugment with Multi-Agent Collaborationcited by 8
  28. 2022Interaction Pattern Disentangling for Multi-Agent Reinforcement Learningcited by 8
  29. 2024eQMARL: Entangled Quantum Multi-Agent Reinforcement Learning for Distributed Cooperation over Quantum Channelscited by 7
  30. 2025Exponential Topology-enabled Scalable Communication in Multi-agent Reinforcement Learningcited by 7
  31. 2022Effects of Spectral Normalization in Multi-agent Reinforcement Learningcited by 6
  32. 2023SMAClite: A Lightweight Environment for Multi-Agent Reinforcement Learningcited by 6
  33. 2023Multi-Agent Reinforcement Learning Guided by Signal Temporal Logic Specificationscited by 6
  34. 2023Learning Decentralized Partially Observable Mean Field Control for Artificial Collective Behaviorcited by 6
  35. 2024Settling Decentralized Multi-Agent Coordinated Exploration by Novelty Sharingcited by 6
  36. 2025QLLM: Do We Really Need a Mixing Network for Credit Assignment in Multi-Agent Reinforcement Learning?cited by 6
  37. 2022Learning Task Embeddings for Teamwork Adaptation in Multi-Agent Reinforcement Learningcited by 5
  38. 2022MA2QL: A Minimalist Approach to Fully Decentralized Multi-Agent Reinforcement Learningcited by 5

50 most cited of 207 All 207 (JSON)