mlsanityData to 13 Oct 2026

Multi-agent Reinforcement Learning: A Comprehensive Survey

arXiv 2312.10256 ↗First posted 15 Dec 2023cs.MA cs.AI cs.LGcited by 30

Not on the map. Why?

Abstract (arXiv)

Multi-agent systems (MAS) are widely prevalent and crucially important in numerous real-world applications, where multiple agents must make decisions to achieve their objectives in a shared environment. Despite their ubiquity, the development of intelligent decision-making agents in MAS poses several open challenges to their effective implementation. This survey examines these challenges, placing an emphasis on studying seminal concepts from game theory (GT) and machine learning (ML) and connecting them to recent advancements in multi-agent reinforcement learning (MARL), i.e. the research of data-driven decision-making within MAS. Therefore, the objective of this survey is to provide a comprehensive perspective along the various dimensions of MARL, shedding light on the unique opportunities that are presented in MARL applications while highlighting the inherent challenges that accompany this potential. Therefore, we hope that our work will not only contribute to the field by analyzing the current landscape of MARL but also motivate future directions with insights for deeper integration of concepts from related domains of GT and ML. With this in mind, this work delves into a detailed exploration of recent and past efforts of MARL and its related fields and describes prior solutions that were proposed and their limitations, as well as their applications.

Key facts. Cited by 30 papers in the map (data through 13 Oct 2026). Builds on 130 of its 244 references that are in the map. First posted on arXiv 15 Dec 2023.

Builds on

130 of 244 references.

References · 130

  1. 2017Proximal Policy Optimization Algorithmscited by 14,592
  2. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,318
  3. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,656
  4. 2018How Powerful are Graph Neural Networks?cited by 4,373
  5. 2015Continuous control with deep reinforcement learningcited by 3,290
  6. 2013Playing Atari with Deep Reinforcement Learningcited by 3,221
  7. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,310
  8. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 1,940
  9. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,889
  10. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,782
  11. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,710
  12. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,620
38 more
  1. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,525
  2. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,257
  3. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,185
  4. 2017Hindsight Experience Replaycited by 1,104
  5. 2015Prioritized Experience Replaycited by 1,078
  6. 2021The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Gamescited by 910
  7. 2021Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learningcited by 890
  8. 2018Representation Learning on Graphs with Jumping Knowledge Networkscited by 877
  9. 2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 871
  10. 2017Rainbow: Combining Improvements in Deep Reinforcement Learningcited by 777
  11. 2018Recurrent World Models Facilitate Policy Evolutioncited by 766
  12. 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Modelscited by 754
  13. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 747
  14. 2017A Distributional Perspective on Reinforcement Learningcited by 745
  15. 2017Counterfactual Multi-Agent Policy Gradientscited by 739
  16. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 731
  17. 2018QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 700
  18. 2019The StarCraft Multi-Agent Challengecited by 592
  19. 2016Learning to Communicate with Deep Multi-Agent Reinforcement Learningcited by 566
  20. 2015Weight Uncertainty in Neural Networkscited by 554
  21. 2017Value-Decomposition Networks For Cooperative Multi-Agent Learningcited by 493
  22. 2017Distributional Reinforcement Learning with Quantile Regressioncited by 465
  23. 2015Deep Recurrent Q-Learning for Partially Observable MDPscited by 459
  24. 2016Reinforcement Learning with Unsupervised Auxiliary Taskscited by 403
  25. 2021Brax -- A Differentiable Physics Engine for Large Scale Rigid Body Simulationcited by 379
  26. 2018Implicit Quantile Networks for Distributional Reinforcement Learningcited by 363
  27. 2016Learning Multiagent Communication with Backpropagationcited by 359
  28. 2019On the Utility of Learning about Humans for Human-AI Coordinationcited by 346
  29. 2016Safe, Multi-Agent, Reinforcement Learning for Autonomous Drivingcited by 302
  30. 2020Agent57: Outperforming the Atari Human Benchmarkcited by 302
  31. 2015Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Imagescited by 270
  32. 2017A Unified Game-Theoretic Approach to Multiagent Reinforcement Learningcited by 265
  33. 2017Noisy Networks for Explorationcited by 264
  34. 2020Decoupling Representation Learning from Reinforcement Learningcited by 264
  35. 2018Human-level performance in first-person multiplayer games with population-based deep reinforcement learningcited by 260
  36. 2019Google Research Football: A Novel Reinforcement Learning Environmentcited by 258
  37. 2017Concrete Dropoutcited by 227
  38. 2013The Complexity of Decentralized Control of Markov Decision Processescited by 226

50 most cited of 130 All 130 (JSON)

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2024: 3; 2025: 17; 2026: 10.

30 papers.

Sort:

By area:

  1. 2025The Landscape of Agentic Reinforcement Learning for LLMs: A Surveycited by 200
  2. 2025A Survey of Reinforcement Learning for Large Reasoning Modelscited by 177
  3. 2025A Survey of Context Engineering for Large Language Modelscited by 122
  4. 2024LLM as a Mastermind: A Survey of Strategic Reasoning with Large Language Modelscited by 98
  5. 2025GAM-Agent: Game-Theoretic and Uncertainty-Aware Collaboration for Complex Visual Reasoningcited by 57
  6. 2025LLM-Powered Decentralized Generative Agents with Adaptive Hierarchical Knowledge Graph for Cooperative Planningcited by 17
  7. 2025AgentGroupChat-V2: Divide-and-Conquer Is What LLM-Based Multi-Agent System Needcited by 12
  8. 2026Generative AI collective behavior needs an interactionist paradigmcited by 6
  9. 2025Evaluating LLMs in Open-Source Gamescited by 5
  10. 2024CANDID DAC: Leveraging Coupled Action Dimensions with Importance Differences in DACcited by 4
  11. 2024Representation Learning For Efficient Deep Multi-Agent Reinforcement Learningcited by 3
  12. 2025Skill-Aligned Fairness in Multi-Agent Learning for Collaboration in Healthcarecited by 3
18 more
  1. 2025A Principle of Targeted Intervention for Multi-Agent Reinforcement Learningcited by 3
  2. 2026CRAFT: Grounded Multi-Agent Coordination Under Partial Informationcited by 3
  3. 2026Building Better Environments for Autonomous Cyber Defencecited by 3
  4. 2025Hierarchical Message-Passing Policies for Multi-Agent Reinforcement Learningcited by 2
  5. 2025Multi-Agent Reinforcement Learning in Intelligent Transportation Systems: A Comprehensive Surveycited by 2
  6. 2026Modularized Reinforcement Learning on LLMs: From MDP Creation to Exploration and Learningcited by 2
  7. 2025Multi-agent Auto-Bidding with Latent Graph Diffusion Modelscited by 1
  8. 2025Multi-Agent Reinforcement Learning in Cybersecurity: From Fundamentals to Applicationscited by 1
  9. 2026Safe Heterogeneous Multi-Agent RL with Communication Regularization for Coordinated Target Acquisitioncited by 1
  10. 2026MACRO-LLM: LLM-Empowered Multi-Agent Collaborative Reasoning under Spatiotemporal Partial Observabilitycited by 1
  11. 2025MAGIC-MASK: Multi-Agent Guided Inter-Agent Collaboration with Mask-Based Explainability for Reinforcement Learningcited by 0
  12. 2025Distributed Area Coverage with High Altitude Balloons Using Multi-Agent Reinforcement Learningcited by 0
  13. 2025Persona-based Multi-Agent Collaboration for Brainstormingcited by 0
  14. 2025MaRCA: Multi-Agent Reinforcement Learning for Dynamic Computation Allocation in Large-Scale Recommender Systemscited by 0
  15. 2026Co-jump: Cooperative Jumping with Quadrupedal Robots via Multi-Agent Reinforcement Learningcited by 0
  16. 2026Subjective-Graph LLM Agents for Simulating Uncertainty in Classroom Social Perceptioncited by 0
  17. 2026Multi-Agent Systems: From Classical Paradigms to Large Foundation Model-Enabled Futurescited by 0
  18. 2026TRACER: Turn-level Regret Matching with Inner Reinforcement Credit for Cooperative Multi-LLM Reasoningcited by 0