# GCS: Graph-based Coordination Strategy for Multi-Agent Reinforcement Learning

arXiv: https://arxiv.org/abs/2201.06257 · HTML: https://mlsanity.com/paper/2201.06257/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Jan 2022.

## Built on by (most cited first, 20 of 20)

- [A Survey of Frontiers in LLM Reasoning: Inference Scaling, Learning to Reason, and Agentic Systems](https://mlsanity.com/paper/2504.09037/) (2025)
- [ACE: Cooperative Multi-agent Q-learning with Bidirectional Action-Dependency](https://mlsanity.com/paper/2211.16068/) (2022)
- [Enhancing Multi-Agent Systems via Reinforcement Learning with LLM-based Planner and Graph-based Policy](https://mlsanity.com/paper/2503.10049/) (2025)
- [Adaptive action supervision in reinforcement learning from real-world multi-agent demonstrations](https://mlsanity.com/paper/2305.13030/) (2023)
- [Scalable Model-based Policy Optimization for Decentralized Networked Systems](https://mlsanity.com/paper/2207.06559/) (2022)
- [Partially Observable Mean Field Multi-Agent Reinforcement Learning Based on Graph-Attention](https://mlsanity.com/paper/2304.12653/) (2023)
- [Learning Graph-Enhanced Commander-Executor for Multi-Agent Navigation](https://mlsanity.com/paper/2302.04094/) (2023)
- [AC2C: Adaptively Controlled Two-Hop Communication for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2302.12515/) (2023)
- [An Empirical Study on Google Research Football Multi-agent Scenarios](https://mlsanity.com/paper/2305.09458/) (2023)
- [Mixture of personality improved Spiking actor network for efficient multi-agent cooperation](https://mlsanity.com/paper/2305.05898/) (2023)
- [Enhancing Cooperative Multi-Agent Reinforcement Learning with State Modelling and Adversarial Exploration](https://mlsanity.com/paper/2505.05262/) (2025)
- [Context-Aware Bayesian Network Actor-Critic Methods for Cooperative Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2306.01920/) (2023)
- [Contextual Knowledge Sharing in Multi-Agent Reinforcement Learning with Decentralized Communication and Coordination](https://mlsanity.com/paper/2501.15695/) (2025)
- [Action-Graph Policies: Learning Action Co-dependencies in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2602.17009/) (2026)
- [AOAD-MAT: Transformer-based multi-agent deep reinforcement learning model considering agents' order of action decisions](https://mlsanity.com/paper/2510.13343/) (2025)
- [Learning to Advise and Learning from Advice in Cooperative Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2205.11163/) (2022)
- [Staged Reinforcement Learning for Complex Tasks through Decomposed Environments](https://mlsanity.com/paper/2311.02746/) (2023)
- [Action Dependency Graphs for Globally Optimal Coordinated Reinforcement Learning](https://mlsanity.com/paper/2506.00797/) (2025)
- [Goal-Oriented Multi-Agent Reinforcement Learning for Decentralized Agent Teams](https://mlsanity.com/paper/2511.11992/) (2025)
- [Do LLM-derived graph priors improve multi-agent coordination?](https://mlsanity.com/paper/2604.17191/) (2026)

## Abstract (arXiv)

Many real-world scenarios involve a team of agents that have to coordinate their policies to achieve a shared goal. Previous studies mainly focus on decentralized control to maximize a common reward and barely consider the coordination among control policies, which is critical in dynamic and complicated environments. In this work, we propose factorizing the joint team policy into a graph generator and graph-based coordinated policy to enable coordinated behaviours among agents. The graph generator adopts an encoder-decoder framework that outputs directed acyclic graphs (DAGs) to capture the underlying dynamic decision structure. We also apply the DAGness-constrained and DAG depth-constrained optimization in the graph generator to balance efficiency and performance. The graph-based coordinated policy exploits the generated decision structure. The graph generator and coordinated policy are trained simultaneously to maximize the discounted return. Empirical evaluations on Collaborative Gaussian Squeeze, Cooperative Navigation, and Google Research Football demonstrate the superiority of the proposed method.