mlsanityData to 7 Oct 2026

A Survey of Multi-Agent Deep Reinforcement Learning with Communication

arXiv 2203.08975 ↗First posted 16 Mar 2022cs.MA cs.LGcited by 103

Not on the map. Why?

Cited by 21 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Communication is an effective mechanism for coordinating the behaviors of multiple agents, broadening their views of the environment, and to support their collaborations. In the field of multi-agent deep reinforcement learning (MADRL), agents can improve the overall learning performance and achieve their objectives by communication. Agents can communicate various types of messages, either to all agents or to specific agent groups, or conditioned on specific constraints. With the growing body of research work in MADRL with communication (Comm-MADRL), there is a lack of a systematic and structural approach to distinguish and classify existing Comm-MADRL approaches. In this paper, we survey recent works in the Comm-MADRL field and consider various aspects of communication that can play a role in designing and developing multi-agent reinforcement learning systems. With these aspects in mind, we propose 9 dimensions along which Comm-MADRL approaches can be analyzed, developed, and compared. By projecting existing works into the multi-dimensional space, we discover interesting trends. We also propose some novel directions for designing future Comm-MADRL systems through exploring possible combinations of the dimensions.

Key facts. Cited by 103 papers in the map (data through 7 Oct 2026). Builds on 68 of its 120 references that are in the map. First posted on arXiv 16 Mar 2022.

Builds on

68 of 120 references.

References · 68

  1. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,708
  2. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 1,916
  3. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,519
  4. 2017Multimodal Machine Learning: A Survey and Taxonomycited by 1,004
  5. 2017Counterfactual Multi-Agent Policy Gradientscited by 733
  6. 2018QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 697
  7. 2019The StarCraft Multi-Agent Challengecited by 589
  8. 2016Learning to Communicate with Deep Multi-Agent Reinforcement Learningcited by 561
  9. 2019Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithmscited by 555
  10. 2017Robust Adversarial Reinforcement Learningcited by 439
  11. 2019QTRAN: Learning to Factorize with Transformation for Cooperative Multi-Agent Reinforcement Learningcited by 372
  12. 2016Learning Multiagent Communication with Backpropagationcited by 355
38 more
  1. 2016Safe, Multi-Agent, Reinforcement Learning for Autonomous Drivingcited by 301
  2. 2017StarCraft II: A New Challenge for Reinforcement Learningcited by 268
  3. 2020QPLEX: Duplex Dueling Multi-Agent Q-Learningcited by 267
  4. 2017Emergence of Grounded Compositional Language in Multi-Agent Populationscited by 257
  5. 2019Google Research Football: A Novel Reinforcement Learning Environmentcited by 257
  6. 2020Benchmarking Multi-Agent Deep Reinforcement Learning Algorithms in Cooperative Taskscited by 207
  7. 2018TarMAC: Targeted Multi-Agent Communicationcited by 201
  8. 2015Multiagent Cooperation and Competition with Deep Reinforcement Learningcited by 180
  9. 2018A Survey and Critique of Multiagent Deep Reinforcement Learningcited by 173
  10. 2018Learning Attentional Communication for Multi-Agent Cooperationcited by 170
  11. 2020Emergent Multi-Agent Communication in the Deep Learning Eracited by 168
  12. 2017Robust Deep Reinforcement Learning with Adversarial Attackscited by 152
  13. 2018Graph Convolutional Reinforcement Learningcited by 148
  14. 2019A Review of Cooperative Multi-Agent Deep Reinforcement Learningcited by 147
  15. 2017Emergence of Language with Multi-agent Games: Learning to Communicate with Sequences of Symbolscited by 123
  16. 2017Stabilising Experience Replay for Deep Multi-Agent Reinforcement Learningcited by 114
  17. 2018Learning when to Communicate at Scale in Multiagent Cooperative and Competitive Taskscited by 114
  18. 2020Compositionality and Generalization in Emergent Languagescited by 103
  19. 2020Off-Policy Multi-Agent Decomposed Policy Gradientscited by 97
  20. 2019Dealing with Non-Stationarity in Multi-Agent Deep Reinforcement Learningcited by 90
  21. 2019Learning to Schedule Communication in Multi-agent Reinforcement Learningcited by 89
  22. 2019Learning Nearly Decomposable Value Functions Via Communication Minimizationcited by 85
  23. 2019Multi-Agent Game Abstraction via Graph Attention Neural Networkcited by 81
  24. 2019On the Pitfalls of Measuring Emergent Communicationcited by 66
  25. 2018Emergent Communication through Negotiationcited by 62
  26. 2019Learning Efficient Multi-agent Communication: An Information Bottleneck Approachcited by 60
  27. 2019Anti-efficient encoding in emergent communicationcited by 59
  28. 2020Learning Individually Inferred Communication for Multi-Agent Cooperationcited by 55
  29. 2020Multi-agent Reinforcement Learning for Networked System Controlcited by 51
  30. 2019Biases for Emergent Communication in Multi-agent Reinforcement Learningcited by 48
  31. 2019Efficient Communication in Multi-Agent Reinforcement Learning via Variance Based Controlcited by 46
  32. 2019Learning Transferable Cooperative Behavior in Multi-Agent Teamscited by 40
  33. 2022Geometric Multimodal Contrastive Representation Learningcited by 40
  34. 2021Learning to Ground Multi-Agent Communication with Autoencoderscited by 37
  35. 2019Learning Agent Communication under Limited Bandwidth by Message Pruningcited by 31
  36. 2019Capacity, Bandwidth, and Compositionality in Emergent Language Learningcited by 30
  37. 2020Succinct and Robust Multi-Agent Communication With Temporal Message Controlcited by 29
  38. 2020Learning Structured Communication for Multi-agent Reinforcement Learningcited by 21

50 most cited of 68 All 68 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 6; 2023: 14; 2024: 17; 2025: 35; 2026: 31.

103 papers.

Sort:

By area:

  1. 2024FinCon: A Synthesized LLM Multi-Agent System with Conceptual Verbal Reinforcement for Enhanced Financial Decision Makingcited by 112
  2. 2024Multi-Agent Reinforcement Learning for Autonomous Driving: A Surveycited by 30
  3. 2024A Survey on Context-Aware Multi-Agent Systems: Techniques, Challenges and Future Directionscited by 27
  4. 2025We Should Identify and Mitigate Third-Party Safety Risks in MCP-Powered Agent Systemscited by 27
  5. 2022AI for Global Climate Cooperation: Modeling Global Climate Negotiations, Agreements, and Long-Term Cooperation in RICE-Ncited by 22
  6. 2024Language Grounded Multi-agent Reinforcement Learning with Human-interpretable Communicationcited by 17
  7. 2026Intelligent AI Delegationcited by 16
  8. 2023Robust multi-agent coordination via evolutionary generation of auxiliary adversarial attackerscited by 15
  9. 2024Generative Emergent Communication: Large Language Model is a Collective World Modelcited by 15
  10. 2025Multi-Agent Reinforcement Learning for Resources Allocation Optimization: A Surveycited by 14
  11. 2025Graph-Augmented Large Language Model Agents: Current Progress and Future Prospectscited by 14
  12. 2024CarDreamer: Open-Source Learning Platform for World Model based Autonomous Drivingcited by 13
38 more
  1. 2024Emergent Language: A Survey and Taxonomycited by 12
  2. 2026OFA-MAS: One-for-All Multi-Agent System Topology Design based on Mixture-of-Experts Graph Generative Modelscited by 12
  3. 2025The Collaboration Gap: Exploration and Benchmarking of Open-World Agentic Cooperationcited by 11
  4. 2023Multi-agent Continual Coordination via Progressive Task Contextualizationcited by 10
  5. 2024Sable: a Performant, Efficient and Scalable Sequence Model for MARLcited by 10
  6. 2026The Five Ws of Multi-Agent Communication: Who Talks to Whom, When, What, and Why -- A Survey from MARL to Emergent Language and LLMscited by 10
  7. 2022DACOM: Learning Delay-Aware Communication for Multi-Agent Reinforcement Learningcited by 9
  8. 2023Emergent Communication in Multi-Agent Reinforcement Learning for Future Wireless Networkscited by 9
  9. 2024Safety-Oriented Pruning and Interpretation of Reinforcement Learning Policiescited by 9
  10. 2025C-3PO: Compact Plug-and-Play Proxy Optimization to Achieve Human-like Retrieval-Augmented Generationcited by 9
  11. 2025Building A Secure Agentic AI Application Leveraging A2A Protocolcited by 9
  12. 2025Orchestrating Human-AI Teams: The Manager Agent as a Unifying Research Challengecited by 8
  13. 2025Explainable and Fine-Grained Safeguarding of LLM Multi-Agent Systems via Bi-Level Graph Anomaly Detectioncited by 8
  14. 2023Towards Semantic Communication Protocols for 6G: From Protocol Learning to Language-Oriented Approachescited by 7
  15. 2025Exponential Topology-enabled Scalable Communication in Multi-agent Reinforcement Learningcited by 7
  16. 2022Enabling the Wireless Metaverse via Semantic Multiverse Communicationcited by 6
  17. 2023Communication-Robust Multi-Agent Learning by Adaptable Auxiliary Multi-Agent Adversary Generationcited by 6
  18. 2022Cooperative Multi-Agent Deep Reinforcement Learning for Reliable and Energy-Efficient Mobile Access via Multi-UAV Controlcited by 5
  19. 2024Cooperation and Fairness in Multi-Agent Reinforcement Learningcited by 5
  20. 2025A Cascading Cooperative Multi-agent Framework for On-ramp Merging Control Integrating Large Language Modelscited by 5
  21. 2024ComTraQ-MPC: Meta-Trained DQN-MPC Integration for Trajectory Tracking with Limited Active Localization Updatescited by 4
  22. 2024A Taxonomy of Architecture Options for Foundation Model-based Agents: Analysis and Decision Modelcited by 4
  23. 2023SoK: Adversarial Machine Learning Attacks and Defences in Multi-Agent Reinforcement Learningcited by 3
  24. 2023CAMMARL: Conformal Action Modeling in Multi Agent Reinforcement Learningcited by 3
  25. 2023Selectively Sharing Experiences Improves Multi-Agent Reinforcement Learningcited by 3
  26. 2025Enhancing Cooperative Multi-Agent Reinforcement Learning with State Modelling and Adversarial Explorationcited by 3
  27. 2025In-Context Reinforcement Learning via Communicative World Modelscited by 3
  28. 2025Multi-Agent Reinforcement Learning with Communication-Constrained Priorscited by 3
  29. 2022Universally Expressive Communication in Multi-Agent Reinforcement Learningcited by 2
  30. 2022Centralized Training with Hybrid Execution in Multi-Agent Reinforcement Learningcited by 2
  31. 2023Efficient Communication via Self-supervised Information Aggregation for Online and Offline Multi-agent Reinforcement Learningcited by 2
  32. 2025Co-Activation Graph Analysis of Safety-Verified and Explainable Deep Reinforcement Learning Policiescited by 2
  33. 2025Reward-Independent Messaging for Decentralized Multi-Agent Reinforcement Learningcited by 2
  34. 2025Decentralized Consensus Inference-based Hierarchical Reinforcement Learning for Multi-Constrained UAV Pursuit-Evasion Gamecited by 2
  35. 2025MARS: Co-evolving Dual-System Deep Research via Multi-Agent Reinforcement Learningcited by 2
  36. 2026LLM-Guided Communication for Cooperative Multi-Agent Reinforcement Learningcited by 2
  37. 2026Modularized Reinforcement Learning on LLMs: From MDP Creation to Exploration and Learningcited by 2
  38. 2023Dynamic Size Message Scheduling for Multi-Agent Communication under Limited Bandwidthcited by 1

50 most cited of 103 All 103 (JSON)