mlsanityData to 14 Oct 2026

A Review of Cooperative Multi-Agent Deep Reinforcement Learning

arXiv 1908.03963 ↗First posted 11 Aug 2019cs.LG cs.AI cs.MAcited by 147

Not on the map. Why?

Abstract (arXiv)

Deep Reinforcement Learning has made significant progress in multi-agent systems in recent years. In this review article, we have focused on presenting recent approaches on Multi-Agent Reinforcement Learning (MARL) algorithms. In particular, we have focused on five common approaches on modeling and solving cooperative multi-agent reinforcement learning problems: (I) independent learners, (II) fully observable critic, (III) value function factorization, (IV) consensus, and (IV) learn to communicate. First, we elaborate on each of these methods, possible challenges, and how these challenges were mitigated in the relevant papers. If applicable, we further make a connection among different papers in each category. Next, we cover some new emerging research areas in MARL along with the relevant recent papers. Due to the recent success of MARL in real-world applications, we assign a section to provide a review of these applications and corresponding articles. Also, a list of available environments for MARL research is provided in this survey. Finally, the paper is concluded with proposals on the possible research directions.

Key facts. Cited by 147 papers in the map (data through 14 Oct 2026). First posted on arXiv 11 Aug 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 19; 2022: 19; 2023: 33; 2024: 17; 2025: 34; 2026: 25.

147 papers.

Sort:

By area:

  1. 2022A Survey of Multi-Agent Deep Reinforcement Learning with Communicationcited by 104
  2. 2023Collaborative Perception in Autonomous Driving: Methods, Datasets and Challengescited by 87
  3. 2024Embodied LLM Agents Learn to Cooperate in Organized Teamscited by 82
  4. 2021Federated Reinforcement Learning: Techniques, Applications, and Open Challengescited by 81
  5. 2022Towards a Standardised Performance Evaluation Protocol for Cooperative MARLcited by 49
  6. 2021Efficient Model-Based Multi-Agent Mean-Field Reinforcement Learningcited by 38
  7. 2022COLA: Consistent Learning with Opponent-Learning Awarenesscited by 38
  8. 2021Independent Natural Policy Gradient Always Converges in Markov Potential Gamescited by 37
  9. 2022Multi-agent Dynamic Algorithm Configurationcited by 36
  10. 2025Multi-agent Embodied AI: Advances and Future Directionscited by 36
  11. 2022Robust Task Representations for Offline Meta-Reinforcement Learning via Contrastive Learningcited by 35
  12. 2021Gradient play in stochastic games: stationary points, convergence, and sample complexitycited by 34
38 more
  1. 2022MASER: Multi-Agent Reinforcement Learning with Subgoals Generated from Experience Replay Buffercited by 34
  2. 2024Coevolving with the Other You: Fine-Tuning LLM with Sequential Cooperative Multi-Agent Reinforcement Learningcited by 33
  3. 2021Continuous Coordination As a Realistic Scenario for Lifelong Learningcited by 32
  4. 2021Deep Reinforcement Learning Versus Evolution Strategies: A Comparative Surveycited by 28
  5. 2021Can Reinforcement Learning Find Stackelberg-Nash Equilibria in General-Sum Markov Games with Myopic Followers?cited by 24
  6. 2024A social path to human-like artificial intelligencecited by 20
  7. 2024AdaSociety: An Adaptive Environment with Social Structures for Multi-Agent Decision-Makingcited by 20
  8. 2021Provably Efficient Multi-Task Reinforcement Learning with Model Transfercited by 16
  9. 2022An Introduction to Multi-Agent Reinforcement Learning and Review of its Application to Autonomous Mobilitycited by 16
  10. 2022Distributed Reinforcement Learning for Robot Teams: A Reviewcited by 16
  11. 2021Self-Organized Polynomial-Time Coordination Graphscited by 15
  12. 2022Robustness and Adaptability of Reinforcement Learning based Cooperative Autonomous Driving in Mixed-autonomy Trafficcited by 15
  13. 2023Provably Efficient Generalized Lagrangian Policy Optimization for Safe Multi-Agent Reinforcement Learningcited by 14
  14. 2023Transformers in Reinforcement Learning: A Surveycited by 12
  15. 2025Enhancing Multi-Agent Systems via Reinforcement Learning with LLM-based Planner and Graph-based Policycited by 12
  16. 2023Online Control Barrier Functions for Decentralized Multi-Agent Navigationcited by 11
  17. 2023Multi-agent Continual Coordination via Progressive Task Contextualizationcited by 11
  18. 2025M3HF: Multi-agent Reinforcement Learning from Multi-phase Human Feedback of Mixed Qualitycited by 11
  19. 2025MEAL: A Benchmark for Continual Multi-Agent Reinforcement Learningcited by 11
  20. 2023Efficient Adversarial Attacks on Online Multi-agent Reinforcement Learningcited by 10
  21. 2021LINDA: Multi-Agent Local Information Decomposition for Awareness of Teammatescited by 9
  22. 2023IMP-MARL: a Suite of Environments for Large-scale Infrastructure Management Planning via MARLcited by 9
  23. 2025Latent Theory of Mind: A Decentralized Diffusion Architecture for Cooperative Manipulationcited by 9
  24. 2025Neural cellular automata: applications to biology and beyond classical AIcited by 9
  25. 2023Human Choice Prediction in Language-based Persuasion Games: Simulation-based Off-Policy Evaluationcited by 8
  26. 2026MO-MIX: Multi-Objective Multi-Agent Cooperative Decision-Making With Deep Reinforcement Learningcited by 8
  27. 2021Attention Actor-Critic algorithm for Multi-Agent Constrained Co-operative Reinforcement Learningcited by 7
  28. 2021Coordinated Reinforcement Learning for Optimizing Mobile Networkscited by 7
  29. 2022CTDS: Centralized Teacher with Decentralized Student for Multi-Agent Reinforcement Learningcited by 7
  30. 2023Self-Motivated Multi-Agent Explorationcited by 7
  31. 2024Inferring Latent Temporal Sparse Coordination Graph for Multi-Agent Reinforcement Learningcited by 7
  32. 2025RALLY: Role-Adaptive LLM-Driven Yoked Navigation for Agentic UAV Swarmscited by 7
  33. 2025CREW-WILDFIRE: Benchmarking Agentic Multi-Agent Collaborations at Scalecited by 7
  34. 2022Distributed Multi-Agent Reinforcement Learning Based on Graph-Induced Local Value Functionscited by 6
  35. 2022Agent-Time Attention for Sparse Rewards Multi-Agent Reinforcement Learningcited by 6
  36. 2022E-MAPP: Efficient Multi-Agent Reinforcement Learning with Parallel Program Guidancecited by 6
  37. 2023Communication-Robust Multi-Agent Learning by Adaptable Auxiliary Multi-Agent Adversary Generationcited by 6
  38. 2023SACHA: Soft Actor-Critic with Heuristic-Based Attention for Partially Observable Multi-Agent Path Findingcited by 6

50 most cited of 147 All 147 (JSON)