mlsanityData to 14 Oct 2026

MARL-GPT: Foundation Model for Multi-Agent Reinforcement Learning

arXiv 2604.05943 ↗First posted 7 Apr 2026cs.AIcited by 0

On the map: LLM agents and tool use → Reinforcement learning for LLM agents

Abstract (arXiv)

Recent advances in multi-agent reinforcement learning (MARL) have demonstrated success in numerous challenging domains and environments, but typically require specialized models for each task. In this work, we propose a coherent methodology that makes it possible for a single GPT-based model to learn and perform well across diverse MARL environments and tasks, including StarCraft Multi-Agent Challenge, Google Research Football and POGEMA. Our method, MARL-GPT, applies offline reinforcement learning to train at scale on the expert trajectories (400M for SMACv2, 100M for GRF, and 1B for POGEMA) combined with a single transformer-based observation encoder that requires no task-specific tuning. Experiments show that MARL-GPT achieves competitive performance compared to specialized baselines in all tested environments. Thus, our findings suggest that it is, indeed, possible to build a multi-task transformer-based model for a wide variety of (significantly different) multi-agent problems paving the way to the fundamental MARL model (akin to ChatGPT, Llama, Mistral etc. in natural language modeling).

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Reinforcement learning for LLM agents, one of 16 directions in LLM agents and tool use. Builds on 22 of its 50 references that are in the map. First posted on arXiv 7 Apr 2026.

Builds on

22 of 50 references.

References · 22

  1. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  2. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,546
  3. 2024Octo: An Open-Source Generalist Robot Policycited by 1,484
  4. 2019The StarCraft Multi-Agent Challengecited by 597
  5. 2019Google Research Football: A Novel Reinforcement Learning Environmentcited by 260
  6. 2024Magentic-One: A Generalist Multi-Agent System for Solving Complex Taskscited by 232
  7. 2023Foundation Models for Decision Making: Problems, Methods, and Opportunitiescited by 164
  8. 2022Multi-Agent Reinforcement Learning is a Sequence Modeling Problemcited by 155
  9. 2024Stop Regressing: Training Value Functions via Classification for Scalable Deep RLcited by 144
  10. 2023Q-Transformer: Scalable Offline Reinforcement Learning via Autoregressive Q-Functionscited by 131
  11. 2018Multi-Agent Generative Adversarial Imitation Learningcited by 100
  12. 2017Coordinated Multi-Agent Imitation Learningcited by 72
10 more
  1. 2018Multi-Agent Imitation Learning for Driving Simulationcited by 50
  2. 2023SCRIMP: Scalable Communication for Reinforcement- and Imitation-Learning-Based Multi-Agent Pathfindingcited by 37
  3. 2023Recurrent Action Transformer with Memorycited by 17
  4. 2024MAPF-GPT: Imitation Learning for Multi-Agent Pathfinding at Scalecited by 17
  5. 2024AMAGO-2: Breaking the Multi-Task Barrier in Meta-Reinforcement Learning with Transformerscited by 15
  6. 2021Multi-Agent Imitation Learning with Copulascited by 13
  7. 2022Conditional Imitation Learning for Multi-Agent Gamescited by 13
  8. 2024Multi-Agent Imitation Learning: Value is Easy, Regret is Hardcited by 11
  9. 2024Work Smarter Not Harder: Simple Imitation Learning with CS-PIBT Outperforms Large Scale Imitation Learning for MAPFcited by 10
  10. 2025SPECTra: Scalable Multi-Agent Reinforcement Learning with Permutation-Free Networkscited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2021Offline Pre-trained Multi-Agent Decision Transformer: One Big Sequence Model Tackles All SMAC Tasks
  2. 2024Dispelling the Mirage of Progress in Offline MARL through Standardised Baselines and Evaluation
  3. 2024Hybrid Training for Enhanced Multi-task Generalization in Multi-agent Reinforcement Learning
  4. 2026Out-of-Distribution Generalisation with Sequence Models in Offline Multi-Agent Reinforcement Learning
  5. 2025The Heterogeneous Multi-Agent Challenge
  6. 2024MARL-LNS: Cooperative Multi-agent Reinforcement Learning via Large Neighborhoods Search
  7. 2025Language-Driven Coordination and Learning in Multi-Agent Simulation Environments
  8. 2025LAMARL: LLM-Aided Multi-Agent Reinforcement Learning for Cooperative Policy Generation