mlsanityData to 7 Oct 2026

Inferring Latent Temporal Sparse Coordination Graph for Multi-Agent Reinforcement Learning

arXiv 2403.19253 ↗First posted 28 Mar 2024cs.LG cs.MAcited by 5

Not on the map. Why?

Abstract (arXiv)

Effective agent coordination is crucial in cooperative Multi-Agent Reinforcement Learning (MARL). While agent cooperation can be represented by graph structures, prevailing graph learning methods in MARL are limited. They rely solely on one-step observations, neglecting crucial historical experiences, leading to deficient graphs that foster redundant or detrimental information exchanges. Additionally, high computational demands for action-pair calculations in dense graphs impede scalability. To address these challenges, we propose inferring a Latent Temporal Sparse Coordination Graph (LTS-CG) for MARL. The LTS-CG leverages agents' historical observations to calculate an agent-pair probability matrix, where a sparse graph is sampled from and used for knowledge exchange between agents, thereby simultaneously capturing agent dependencies and relation uncertainty. The computational complexity of this procedure is only related to the number of agents. This graph learning process is further augmented by two innovative characteristics: Predict-Future, which enables agents to foresee upcoming observations, and Infer-Present, ensuring a thorough grasp of the environmental context from limited data. These features allow LTS-CG to construct temporal graphs from historical and real-time information, promoting knowledge exchange during policy learning and effective collaboration. Graph learning and agent training occur simultaneously in an end-to-end manner. Our demonstrated results on the StarCraft II benchmark underscore LTS-CG's superior performance.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 35 of its 55 references that are in the map. First posted on arXiv 28 Mar 2024.

Builds on

35 of 55 references.

References · 35

  1. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  2. 2019A Comprehensive Survey on Graph Neural Networkscited by 2,906
  3. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,708
  4. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,519
  5. 2016The Concrete Distribution: A Continuous Relaxation of Discrete Random Variablescited by 1,135
  6. 2017Spatio-Temporal Graph Convolutional Networks: A Deep Learning Framework for Traffic Forecastingcited by 1,125
  7. 2021The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Gamescited by 903
  8. 2017Counterfactual Multi-Agent Policy Gradientscited by 733
  9. 2018QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 697
  10. 2019The StarCraft Multi-Agent Challengecited by 589
  11. 2020Connecting the Dots: Multivariate Time Series Forecasting with Graph Neural Networkscited by 563
  12. 2019QTRAN: Learning to Factorize with Transformation for Cooperative Multi-Agent Reinforcement Learningcited by 372
23 more
  1. 2018Actor-Attention-Critic for Multi-Agent Reinforcement Learningcited by 259
  2. 2019Learning Discrete Structures for Graph Neural Networkscited by 228
  3. 2021Discrete Graph Structure Learning for Forecasting Multiple Time Seriescited by 169
  4. 2018Graph Convolutional Reinforcement Learningcited by 148
  5. 2019A Review of Cooperative Multi-Agent Deep Reinforcement Learningcited by 147
  6. 2020ROMA: Multi-Agent Reinforcement Learning with Emergent Rolescited by 135
  7. 2019Deep Coordination Graphscited by 96
  8. 2019Learning Nearly Decomposable Value Functions Via Communication Minimizationcited by 85
  9. 2019Multi-Agent Game Abstraction via Graph Attention Neural Networkcited by 81
  10. 2020Deep Implicit Coordination Graphs for Multi-agent Reinforcement Learningcited by 58
  11. 2018Multi-Agent Reinforcement Learning Based Resource Allocation for UAV Networkscited by 54
  12. 2019PIC: Permutation Invariant Critic for Multi-Agent Deep Reinforcement Learningcited by 46
  13. 2019Learning to Score Behaviors for Guided Policy Optimizationcited by 34
  14. 2021Context-Aware Sparse Deep Coordination Graphscited by 27
  15. 2021DFAC Framework: Factorizing the Value Function via Quantile Mixture for Multi-Agent Distributional Q-Learningcited by 26
  16. 2022Multivariate Time Series Forecasting with Latent Graph Inferencecited by 20
  17. 2020Graph Convolutional Value Decomposition in Multi-Agent Reinforcement Learningcited by 14
  18. 2021Self-Organized Polynomial-Time Coordination Graphscited by 14
  19. 2022Rethinking Individual Global Max in Cooperative Multi-Agent Reinforcement Learningcited by 13
  20. 2022EvolveHypergraph: Group-Aware Dynamic Relational Reasoning for Trajectory Predictioncited by 10
  21. 2022Learning from the Dark: Boosting Graph Convolutional Neural Networks with Diverse Negative Samplescited by 10
  22. 2021Reducing Bus Bunching with Asynchronous Multi-Agent Reinforcement Learningcited by 7
  23. 2024Layer-diverse Negative Sampling for Graph Neural Networkscited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 2; 2026: 2.

5 papers.

  1. 2024Group-Aware Coordination Graph for Multi-Agent Reinforcement Learningcited by 17
  2. 2025Bayesian Ego-graph Inference for Networked Multi-Agent Reinforcement Learningcited by 2
  3. 2025Hierarchical Message-Passing Policies for Multi-Agent Reinforcement Learningcited by 1
  4. 2026A Survey of Multi-Agent Deep Reinforcement Learning with Graph Neural Network-Based Communicationcited by 0
  5. 2026Heterogeneous Information-Bottleneck Coordination Graphs for Multi-Agent Reinforcement Learningcited by 0