mlsanityData to 7 Oct 2026

Deep Coordination Graphs

arXiv 1910.00091 ↗First posted 27 Sep 2019cs.LG cs.AI cs.MAcited by 96

Not on the map. Why?

Abstract (arXiv)

This paper introduces the deep coordination graph (DCG) for collaborative multi-agent reinforcement learning. DCG strikes a flexible trade-off between representational capacity and generalization by factoring the joint value function of all agents according to a coordination graph into payoffs between pairs of agents. The value can be maximized by local message passing along the graph, which allows training of the value function end-to-end with Q-learning. Payoff functions are approximated with deep neural networks that employ parameter sharing and low-rank approximations to significantly improve sample efficiency. We show that DCG can solve predator-prey tasks that highlight the relative overgeneralization pathology, as well as challenging StarCraft II micromanagement tasks.

Key facts. Cited by 96 papers in the map (data through 7 Oct 2026). First posted on arXiv 27 Sep 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 18; 2022: 23; 2023: 14; 2024: 13; 2025: 16; 2026: 12.

96 papers.

Sort:

By area:

  1. 2021Celebrating Diversity in Shared Multi-Agent Reinforcement Learningcited by 96
  2. 2021Rethinking the Implementation Tricks and Monotonicity Constraint in Cooperative Multi-Agent Reinforcement Learningcited by 68
  3. 2022Towards a Standardised Performance Evaluation Protocol for Cooperative MARLcited by 48
  4. 2021Learning Long-Term Reward Redistribution via Randomized Return Decompositioncited by 33
  5. 2023Multi-agent Reinforcement Learning: A Comprehensive Surveycited by 30
  6. 2021Context-Aware Sparse Deep Coordination Graphscited by 27
  7. 2022Challenges and Opportunities in Deep Reinforcement Learning with Graph Neural Networks: A Comprehensive review of Algorithms and Applicationscited by 27
  8. 2023A Review of Cooperation in Multi-agent Learningcited by 27
  9. 2021SHAQ: Incorporating Shapley Value Theory into Multi-Agent Q-Learningcited by 26
  10. 2021Multi-Agent MDP Homomorphic Networkscited by 24
  11. 2022Maximum Class Separation as Inductive Bias in One Matrixcited by 21
  12. 2022Solving Continuous Control via Q-learningcited by 21
38 more
  1. 2023MAC-PO: Multi-Agent Experience Replay via Collective Priority Optimizationcited by 20
  2. 2023Graph Decision Transformercited by 20
  3. 2022More Centralized Training, Still Decentralized Execution: Multi-Agent Conditional Policy Factorizationcited by 19
  4. 2024AdaSociety: An Adaptive Environment with Social Structures for Multi-Agent Decision-Makingcited by 19
  5. 2021Mis-spoke or mis-lead: Achieving Robustness in Multi-Agent Communicative Reinforcement Learningcited by 18
  6. 2023Is Centralized Training with Decentralized Execution Framework Centralized Enough for MARL?cited by 18
  7. 2024Group-Aware Coordination Graph for Multi-Agent Reinforcement Learningcited by 17
  8. 2022Equivariant Networks for Zero-Shot Coordinationcited by 16
  9. 2022ACE: Cooperative Multi-agent Q-learning with Bidirectional Action-Dependencycited by 15
  10. 2025CoLLMLight: Cooperative Large Language Model Agents for Network-Wide Traffic Signal Controlcited by 15
  11. 2021Self-Organized Polynomial-Time Coordination Graphscited by 14
  12. 2022QGNN: Value Function Factorisation with Graph Neural Networkscited by 14
  13. 2022Multi-Agent Coordination via Multi-Level Communicationcited by 14
  14. 2022Pareto Actor-Critic for Equilibrium Selection in Multi-Agent Reinforcement Learningcited by 14
  15. 2022Collaborative Uncertainty Benefits Multi-Agent Multi-Modal Trajectory Forecastingcited by 13
  16. 2022Value Functions Factorization with Latent State Information Sharing in Decentralized Multi-Agent Policy Gradientscited by 12
  17. 2022A Survey on Large-Population Systems and Scalable Multi-Agent Reinforcement Learningcited by 12
  18. 2022A General Learning Framework for Open Ad Hoc Teamwork Using Graph-based Policy Learningcited by 12
  19. 2023Tackling Cooperative Incompatibility for Zero-Shot Human-AI Coordinationcited by 12
  20. 2021Learning Emergent Discrete Message Communication for Cooperative Reinforcement Learningcited by 11
  21. 2022Non-Linear Coordination Graphscited by 10
  22. 2022Leveraging Smooth Attention Prior for Multi-Agent Trajectory Predictioncited by 9
  23. 2024Multi-Sender Persuasion: A Computational Perspectivecited by 9
  24. 2023Semantically Aligned Task Decomposition in Multi-Agent Reinforcement Learningcited by 8
  25. 2021Coordinated Reinforcement Learning for Optimizing Mobile Networkscited by 7
  26. 2023Partially Observable Mean Field Multi-Agent Reinforcement Learning Based on Graph-Attentioncited by 7
  27. 2022Breaking the Curse of Dimensionality in Multiagent State Space: A Unified Agent Permutation Frameworkcited by 6
  28. 2024Growing Q-Networks: Solving Continuous Control Tasks with Adaptive Control Resolutioncited by 6
  29. 2024Grounded Answers for Multi-agent Decision-making Problem through Generative World Modelcited by 6
  30. 2025BMG-Q: Localized Bipartite Match Graph Attention Q-Learning for Ride-Pooling Order Dispatchcited by 6
  31. 2025Deep Meta Coordination Graphs for Multi-agent Reinforcement Learningcited by 6
  32. 2021Scalable Anytime Planning for Multi-Agent MDPscited by 5
  33. 2021Dealing with Non-Stationarity in MARL via Trust-Region Decompositioncited by 5
  34. 2021Learning to Coordinate via Multiple Graph Neural Networkscited by 5
  35. 2022Learning to Transfer Role Assignment Across Team Sizescited by 5
  36. 2024Fully Decentralized Cooperative Multi-Agent Reinforcement Learning: A Surveycited by 5
  37. 2024AgentMixer: Multi-Agent Correlated Policy Factorizationcited by 5
  38. 2024Inferring Latent Temporal Sparse Coordination Graph for Multi-Agent Reinforcement Learningcited by 5

50 most cited of 96 All 96 (JSON)