mlsanityData to 14 Oct 2026

Do LLM-derived graph priors improve multi-agent coordination?

arXiv 2604.17191 ↗First posted 19 Apr 2026cs.LGcited by 0

On the map: LLM agents and tool use → Reinforcement learning for LLM agents

Abstract (arXiv)

Multi-agent reinforcement learning (MARL) is crucial for AI systems that operate collaboratively in distributed and adversarial settings, particularly in multi-domain operations (MDO). A central challenge in cooperative MARL is determining how agents should coordinate: existing approaches must either hand-specify graph topology, rely on proximity-based heuristics, or learn structure entirely from environment interaction; all of which are brittle, semantically uninformed, or data-intensive. We investigate whether large language models (LLMs) can generate useful coordination graph priors for MARL by using minimal natural language descriptions of agent observations to infer latent coordination patterns. These priors are integrated into MARL algorithms via graph convolutional layers within a graph neural network (GNN)-based pipeline, and evaluated on four cooperative scenarios from the Multi-Agent Particle Environment (MPE) benchmark against baselines spanning the full spectrum of coordination modeling, from independent learners to state-of-the-art graph-based methods. We further ablate across five compact open-source LLMs to assess the sensitivity of prior quality to model choice. Our results provide the first quantitative evidence that LLM-derived graph priors can enhance coordination and adaptability in dynamic multi-agent environments, and demonstrate that models as small as 1.5B parameters are sufficient for effective prior generation.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Reinforcement learning for LLM agents, one of 16 directions in LLM agents and tool use. Builds on 41 of its 56 references that are in the map. First posted on arXiv 19 Apr 2026.

Builds on

41 of 56 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024LLM-based Multi-Agent Reinforcement Learning: Current and Future Directions17 May
  2. 2023Voyager: An Open-Ended Embodied Agent with Large Language Models25 May

Other references · 39

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  3. 2017Graph Attention Networkscited by 8,923
  4. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  5. 2024Qwen2.5 Technical Reportcited by 4,942
  6. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,998
  7. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,558
  8. 2022Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agentscited by 1,035
  9. 2021The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Gamescited by 942
  10. 2023Eureka: Human-Level Reward Design via Coding Large Language Modelscited by 571
  11. 2019Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithmscited by 567
  12. 2017Value-Decomposition Networks For Cooperative Multi-Agent Learningcited by 503
27 more
  1. 2020Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 485
  2. 2019QTRAN: Learning to Factorize with Transformation for Cooperative Multi-Agent Reinforcement Learningcited by 379
  3. 2020QPLEX: Duplex Dueling Multi-Agent Q-Learningcited by 270
  4. 2017Emergence of Grounded Compositional Language in Multi-Agent Populationscited by 263
  5. 2013The Complexity of Decentralized Control of Markov Decision Processescited by 231
  6. 2020Weighted QMIX: Expanding Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 197
  7. 2022Multi-Agent Reinforcement Learning is a Sequence Modeling Problemcited by 155
  8. 2019A Review of Cooperative Multi-Agent Deep Reinforcement Learningcited by 147
  9. 2025Multi-Agent Collaboration via Evolving Orchestrationcited by 108
  10. 2019Deep Coordination Graphscited by 98
  11. 2020Deep Implicit Coordination Graphs for Multi-agent Reinforcement Learningcited by 60
  12. 2024An Introduction to Centralized Training for Decentralized Execution in Cooperative Multi-Agent Reinforcement Learningcited by 50
  13. 2021Context-Aware Sparse Deep Coordination Graphscited by 28
  14. 2022GCS: Graph-based Coordination Strategy for Multi-Agent Reinforcement Learningcited by 20
  15. 2022More Centralized Training, Still Decentralized Execution: Multi-Agent Conditional Policy Factorizationcited by 19
  16. 2024Group-Aware Coordination Graph for Multi-Agent Reinforcement Learningcited by 18
  17. 2021Self-Organized Polynomial-Time Coordination Graphscited by 15
  18. 2021HAMMER: Multi-Level Coordination of Reinforcement Learning Agents via Learned Messagingcited by 13
  19. 2022Non-Linear Coordination Graphscited by 10
  20. 2024Review of Autonomous Mobile Robots for the Warehouse Environmentcited by 7
  21. 2025Deep Meta Coordination Graphs for Multi-agent Reinforcement Learningcited by 6
  22. 2024AgentMixer: Multi-Agent Correlated Policy Factorizationcited by 5
  23. 2023CAMMARL: Conformal Action Modeling in Multi Agent Reinforcement Learningcited by 3
  24. 2025A Framework for Dynamic Situational Awareness in Human Robot Teams: An Interview Studycited by 3
  25. 2025TIGER-MARL: Enhancing Multi-Agent Reinforcement Learning with Temporal Information through Graph-based Embeddings and Representationscited by 3
  26. 2025HierRouter: Coordinated Routing of Specialized Large Language Models via Reinforcement Learningcited by 3
  27. 2026Action-Graph Policies: Learning Action Co-dependencies in Multi-Agent Reinforcement Learningcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Enhancing Multi-Agent Systems via Reinforcement Learning with LLM-based Planner and Graph-based Policy
  2. 2024Inferring Latent Temporal Sparse Coordination Graph for Multi-Agent Reinforcement Learning
  3. 2025Reinforcement Networks: novel framework for collaborative Multi-Agent Reinforcement Learning tasks
  4. 2026AgentGL: Towards Agentic Graph Learning with LLMs via Reinforcement Learning
  5. 2024YOLO-MARL: You Only LLM Once for Multi-Agent Reinforcement Learning
  6. 2025Language-Driven Coordination and Learning in Multi-Agent Simulation Environments
  7. 2026A Survey of Multi-Agent Deep Reinforcement Learning with Graph Neural Network-Based Communication
  8. 2026Coordination Graphs for Constrained Multi-Agent Reinforcement Learning