mlsanityData to 7 Oct 2026

A Survey and Critique of Multiagent Deep Reinforcement Learning

arXiv 1810.05587 ↗First posted 12 Oct 2018cs.MA cs.AI cs.LGcited by 173

Not on the map. Why?

Abstract (arXiv)

Deep reinforcement learning (RL) has achieved outstanding results in recent years. This has led to a dramatic increase in the number of applications and methods. Recent works have explored learning beyond single-agent scenarios and have considered multiagent learning (MAL) scenarios. Initial results report successes in complex multiagent domains, although there are several challenges to be addressed. The primary goal of this article is to provide a clear overview of current multiagent deep reinforcement learning (MDRL) literature. Additionally, we complement the overview with a broader analysis: (i) we revisit previous key components, originally presented in MAL and RL, and highlight how they have been adapted to multiagent deep reinforcement learning settings. (ii) We provide general guidelines to new practitioners in the area: describing lessons learned from MDRL works, pointing to recent benchmarks, and outlining open avenues of research. (iii) We take a more critical tone raising practical challenges of MDRL (e.g., implementation and computational demands). We expect this article will help unify and motivate future research to take advantage of the abundant literature that exists (e.g., RL and MAL) in a joint effort to promote fruitful research in the multiagent community.

Key facts. Cited by 173 papers in the map (data through 7 Oct 2026). First posted on arXiv 12 Oct 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 49; 2022: 30; 2023: 27; 2024: 14; 2025: 25; 2026: 28.

173 papers.

Sort:

By area:

  1. 2022A Survey of Multi-Agent Deep Reinforcement Learning with Communicationcited by 103
  2. 2021From Motor Control to Team Play in Simulated Humanoid Footballcited by 87
  3. 2025MAPoRL: Multi-Agent Post-Co-Training for Collaborative Large Language Models with Reinforcement Learningcited by 80
  4. 2024LLM-based Multi-Agent Reinforcement Learning: Current and Future Directionscited by 67
  5. 2021Safe Multi-Agent Reinforcement Learning via Shieldingcited by 60
  6. 2021Exploration in Deep Reinforcement Learning: From Single-Agent to Multiagent Domaincited by 57
  7. 2021Decentralized Q-Learning in Zero-sum Markov Gamescited by 53
  8. 2021Deep Multiagent Reinforcement Learning: Challenges and Directionscited by 42
  9. 2023Offline Multi-Agent Reinforcement Learning with Implicit Global-to-Local Value Regularizationcited by 42
  10. 2021Efficient Model-Based Multi-Agent Mean-Field Reinforcement Learningcited by 38
  11. 2022Multi-agent Dynamic Algorithm Configurationcited by 36
  12. 2021Learning to Simulate Self-Driven Particles System with Coordinated Policy Optimizationcited by 33
38 more
  1. 2021Diverse Auto-Curriculum is Critical for Successful Real-World Multiagent Learning Systemscited by 24
  2. 2021Can Reinforcement Learning Find Stackelberg-Nash Equilibria in General-Sum Markov Games with Myopic Followers?cited by 24
  3. 2021Deep Reinforcement Learning Versus Evolution Strategies: A Comparative Surveycited by 23
  4. 2022PMIC: Improving Multi-Agent Reinforcement Learning with Progressive Mutual Information Collaborationcited by 23
  5. 2022Law Informs Code: A Legal Informatics Approach to Aligning Artificial Intelligence with Humanscited by 23
  6. 2021A Cooperative-Competitive Multi-Agent Framework for Auto-bidding in Online Advertisingcited by 22
  7. 2025LLM-Powered AI Agent Systems and Their Applications in Industrycited by 22
  8. 2021Expected Value of Communication for Planning in Ad Hoc Teamworkcited by 19
  9. 2021Pervasive AI for IoT applications: A Survey on Resource-efficient Distributed Artificial Intelligencecited by 18
  10. 2021Collaborative Visual Navigationcited by 18
  11. 2021A Survey on Deep Reinforcement Learning for Audio-Based Applicationscited by 17
  12. 2022An Introduction to Multi-Agent Reinforcement Learning and Review of its Application to Autonomous Mobilitycited by 16
  13. 2022Learning to Solve Vehicle Routing Problems: A Surveycited by 16
  14. 2023Robust multi-agent coordination via evolutionary generation of auxiliary adversarial attackerscited by 15
  15. 2024N-Agent Ad Hoc Teamworkcited by 15
  16. 2021Multi-Task Federated Reinforcement Learning with Adversariescited by 13
  17. 2021Recent Advances in Leveraging Human Guidance for Sequential Decision-Making Taskscited by 13
  18. 2023Boundary-aware Supervoxel-level Iteratively Refined Interactive 3D Image Segmentation with Multi-agent Reinforcement Learningcited by 13
  19. 2022Autonomy and Intelligence in the Computing Continuum: Challenges, Enablers, and Future Directions for Orchestrationcited by 12
  20. 2022A Survey on Large-Population Systems and Scalable Multi-Agent Reinforcement Learningcited by 12
  21. 2024Multi-Agent, Human-Agent and Beyond: A Survey on Cooperation in Social Dilemmascited by 12
  22. 2024Emergent Language: A Survey and Taxonomycited by 12
  23. 2021Spatial Intention Maps for Multi-Agent Mobile Manipulationcited by 11
  24. 2022Distributed Reinforcement Learning for Robot Teams: A Reviewcited by 11
  25. 2022Decentralized Optimistic Hyperpolicy Mirror Descent: Provably No-Regret Learning in Markov Gamescited by 11
  26. 2025Evaluating Generalization Capabilities of LLM-Based Agents in Mixed-Motive Scenarios Using Concordiacited by 11
  27. 2022DACOM: Learning Delay-Aware Communication for Multi-Agent Reinforcement Learningcited by 9
  28. 2023A Versatile Multi-Agent Reinforcement Learning Benchmark for Inventory Managementcited by 9
  29. 2024A Multi-Agent Approach for REST API Testing with Semantic Graphs and LLM-Driven Inputscited by 9
  30. 2021Joint Attention for Multi-Agent Coordination and Social Learningcited by 8
  31. 2023Semantically Aligned Task Decomposition in Multi-Agent Reinforcement Learningcited by 8
  32. 2025An Outlook on the Opportunities and Challenges of Multi-Agent AI Systemscited by 8
  33. 2025MEAL: A Benchmark for Continual Multi-Agent Reinforcement Learningcited by 8
  34. 2021Modeling the Interaction between Agents in Cooperative Multi-Agent Reinforcement Learningcited by 7
  35. 2022CTDS: Centralized Teacher with Decentralized Student for Multi-Agent Reinforcement Learningcited by 7
  36. 2023Partially Observable Mean Field Multi-Agent Reinforcement Learning Based on Graph-Attentioncited by 7
  37. 2023Offline Multi-Agent Reinforcement Learning with Coupled Value Factorizationcited by 7
  38. 2023Hybrid Approaches for Moral Value Alignment in AI Agents: a Manifestocited by 7

50 most cited of 173 All 173 (JSON)