mlsanityData to 14 Oct 2026

Learning Multi-Agent Communication from Graph Modeling Perspective

arXiv 2405.08550 ↗First posted 14 May 2024cs.LGcited by 48

Not on the map. Why?

Abstract (arXiv)

In numerous artificial intelligence applications, the collaborative efforts of multiple intelligent agents are imperative for the successful attainment of target objectives. To enhance coordination among these agents, a distributed communication framework is often employed. However, information sharing among all agents proves to be resource-intensive, while the adoption of a manually pre-defined communication architecture imposes limitations on inter-agent communication, thereby constraining the potential for collaborative efforts. In this study, we introduce a novel approach wherein we conceptualize the communication architecture among agents as a learnable graph. We formulate this problem as the task of determining the communication graph while enabling the architecture parameters to update normally, thus necessitating a bi-level optimization process. Utilizing continuous relaxation of the graph representation and incorporating attention units, our proposed approach, CommFormer, efficiently optimizes the communication graph and concurrently refines architectural parameters through gradient descent in an end-to-end manner. Extensive experiments on a variety of cooperative tasks substantiate the robustness of our model across diverse cooperative scenarios, where agents are able to develop more coordinated and sophisticated strategies regardless of changes in the number of agents.

Key facts. Cited by 48 papers in the map (data through 14 Oct 2026). Builds on 40 of its 49 references that are in the map. First posted on arXiv 14 May 2024.

Builds on

40 of 49 references.

References · 40

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,739
  3. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 2,026
  4. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,558
  5. 2016The Concrete Distribution: A Continuous Relaxation of Discrete Random Variablescited by 1,148
  6. 2021The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Gamescited by 942
  7. 2017Counterfactual Multi-Agent Policy Gradientscited by 751
  8. 2019The StarCraft Multi-Agent Challengecited by 597
  9. 2016Learning to Communicate with Deep Multi-Agent Reinforcement Learningcited by 575
  10. 2017Value-Decomposition Networks For Cooperative Multi-Agent Learningcited by 503
  11. 2020Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 485
  12. 2019QTRAN: Learning to Factorize with Transformation for Cooperative Multi-Agent Reinforcement Learningcited by 379
28 more
  1. 2022ST-P3: End-to-end Vision-based Autonomous Driving via Spatial-Temporal Feature Learningcited by 368
  2. 2016Learning Multiagent Communication with Backpropagationcited by 366
  3. 2017A Unified Game-Theoretic Approach to Multiagent Reinforcement Learningcited by 268
  4. 2019Google Research Football: A Novel Reinforcement Learning Environmentcited by 260
  5. 2017Learning with Opponent-Learning Awarenesscited by 222
  6. 2018TarMAC: Targeted Multi-Agent Communicationcited by 208
  7. 2018Mean Field Multi-Agent Reinforcement Learningcited by 207
  8. 2021Trust Region Policy Optimisation in Multi-Agent Reinforcement Learningcited by 186
  9. 2018Learning Attentional Communication for Multi-Agent Cooperationcited by 175
  10. 2022Multi-Agent Reinforcement Learning is a Sequence Modeling Problemcited by 155
  11. 2018Learning when to Communicate at Scale in Multiagent Cooperative and Competitive Taskscited by 121
  12. 2020SMARTS: Scalable Multi-Agent Reinforcement Learning Training School for Autonomous Drivingcited by 121
  13. 2017Multiagent Bidirectionally-Coordinated Nets: Emergence of Human-level Coordination in Learning to Play StarCraft Combat Gamescited by 96
  14. 2020Game-Theoretic Multiagent Reinforcement Learningcited by 94
  15. 2019Learning to Schedule Communication in Multi-agent Reinforcement Learningcited by 93
  16. 2019Learning Nearly Decomposable Value Functions Via Communication Minimizationcited by 86
  17. 2019Graph Transformer for Graph-to-Sequence Learningcited by 82
  18. 2019Multi-Agent Game Abstraction via Graph Attention Neural Networkcited by 82
  19. 2021ToM2C: Target-oriented Multi-agent Communication and Cooperation with Theory of Mindcited by 58
  20. 2019Efficient Ridesharing Order Dispatching with Mean Field Multi-Agent Reinforcement Learningcited by 57
  21. 2021UPDeT: Universal Multi-agent Reinforcement Learning via Policy Decoupling with Transformerscited by 39
  22. 2023Heterogeneous Multi-Robot Reinforcement Learningcited by 36
  23. 2021Coach-Player Multi-Agent Reinforcement Learning for Dynamic Team Compositioncited by 30
  24. 2021MALib: A Parallel Framework for Population-based Multi-agent Reinforcement Learningcited by 23
  25. 2023Graph Decision Transformercited by 20
  26. 2021Towards Collaborative Simultaneous Localization and Mapping: a Survey of the Current Research Landscapecited by 15
  27. 2022QGNN: Value Function Factorisation with Graph Neural Networkscited by 14
  28. 2020Learning Multi-Agent Coordination through Connectivity-driven Communicationcited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 9; 2025: 17; 2026: 22.

48 papers.

Sort:

By area:

  1. 2024Cut the Crap: An Economical Communication Pipeline for LLM-based Multi-Agent Systemscited by 141
  2. 2024G-Designer: Architecting Multi-agent Communication Topologies via Graph Neural Networkscited by 119
  3. 2026A Survey of Agentic Reasoning for Large Language Models: Towards Recursively Self-Improving and Collective Agentscited by 50
  4. 2025Understanding the Information Propagation Effects of Communication Topologies in LLM-based Multi-Agent Systemscited by 43
  5. 2024ResearchTown: Simulator of Human Research Communitycited by 37
  6. 2025Adaptive Graph Pruning for Multi-Agent Communicationcited by 29
  7. 2024Locally Estimated Global Perturbations are Better than Local Perturbations for Federated Sharpness-aware Minimizationcited by 28
  8. 2026From Static Templates to Dynamic Runtime Graphs: A Survey of Workflow Optimization for LLM Agentscited by 24
  9. 2025Graphs Meet AI Agents: Taxonomy, Progress, and Future Opportunitiescited by 23
  10. 2024HarmoDT: Harmony Multi-Task Decision Transformer for Offline Reinforcement Learningcited by 14
  11. 2025Towards Transparent and Incentive-Compatible Collaboration in Decentralized LLM Multi-Agent Systems: A Blockchain-Driven Approachcited by 12
  12. 2025Dynamic Generation of Multi-LLM Agents Communication Topologies with Graph Diffusion Modelscited by 11
36 more
  1. 2025Exponential Topology-enabled Scalable Communication in Multi-agent Reinforcement Learningcited by 7
  2. 2024Prompt Tuning with Diffusion for Few-Shot Pre-trained Policy Generalizationcited by 6
  3. 2025Deep Meta Coordination Graphs for Multi-agent Reinforcement Learningcited by 6
  4. 2025Topological Structure Learning Should Be A Research Priority for LLM-Based Multi-Agent Systemscited by 5
  5. 2026GoAgent: Group-of-Agents Communication Topology Generation for LLM-based Multi-Agent Systemscited by 5
  6. 2026MOC: Multi-Order Communication in LLM-based Multi-Agent Systemscited by 4
  7. 2025A Constrained Multi-Agent Reinforcement Learning Approach to Autonomous Traffic Signal Controlcited by 3
  8. 2025LLM Chemistry Estimation for Multi-LLM Recommendationcited by 3
  9. 2025Multi-Agent Reinforcement Learning with Communication-Constrained Priorscited by 3
  10. 2025Bandwidth-constrained Variational Message Encoding for Cooperative Multi-agent Reinforcement Learningcited by 3
  11. 2026GlobeDiff: State Diffusion Process for Partial Observability in Multi-Agent Systemscited by 3
  12. 2026Retaining Suboptimal Actions to Follow Shifting Optima in Multi-Agent Reinforcement Learningcited by 3
  13. 2025Closed-Loop Vision-Language Planning for Multi-Agent Coordinationcited by 2
  14. 2025Analytic Energy-Guided Policy Optimization for Offline Reinforcement Learningcited by 2
  15. 2025Hierarchical Message-Passing Policies for Multi-Agent Reinforcement Learningcited by 2
  16. 2026Bridging MARL to SARL: An Order-Independent Multi-Agent Transformer via Latent Consensuscited by 2
  17. 2026LLM-Guided Communication for Cooperative Multi-Agent Reinforcement Learningcited by 2
  18. 2024Task-Aware Harmony Multi-Task Decision Transformer for Offline Reinforcement Learningcited by 1
  19. 2024Continual Task Learning through Adaptive Policy Self-Compositioncited by 1
  20. 2025Learning what to say and how precisely: Efficient Communication via Differentiable Discrete Communication Learningcited by 1
  21. 2026Safe Heterogeneous Multi-Agent RL with Communication Regularization for Coordinated Target Acquisitioncited by 1
  22. 2026Differentiable Mixture-of-Agents Incentivizes Swarm Intelligence of Large Language Modelscited by 1
  23. 2026HiComm: Hierarchical Communication for Multi-agent Reinforcement Learningcited by 1
  24. 2024Communication Learning in Multi-Agent Systems from Graph Modeling Perspectivecited by 0
  25. 2025Communication-Enhanced Tutoring for Efficient Decentralized Multi-Agent Reinforcement Learningcited by 0
  26. 2026Conditional Sequence Modeling for Safe Reinforcement Learningcited by 0
  27. 2026SCoUT: Scalable Communication via Utility-Guided Temporal Grouping in Multi-Agent Reinforcement Learningcited by 0
  28. 2026Interference-Aware K-Step Reachable Communication in Multi-Agent Reinforcement Learningcited by 0
  29. 2026Learning Communication Between Heterogeneous Agents in Multi-Agent Reinforcement Learning for Autonomous Cyber Defencecited by 0
  30. 2026Breakthrough the Suboptimal Stable Point in Value-Factorization-Based Multi-Agent Reinforcement Learningcited by 0
  31. 2026Heterogeneous Information-Bottleneck Coordination Graphs for Multi-Agent Reinforcement Learningcited by 0
  32. 2026Decoupling Communication from Policy: Robust MARL under Bandwidth Constraintscited by 0
  33. 2026PEAR: Permutation-Equivariant Adaptive Routing Multi-Agent Debatecited by 0
  34. 2026Communication-Efficient Digital-Twin Coordination for Heterogeneous LLM Embodied Agents over Computing Power Networkscited by 0
  35. 2026Q-learning Penalized Transformer for Safe Offline Reinforcement Learningcited by 0
  36. 2026Continual Graph Multi-Agent Reinforcement Learningcited by 0