mlsanityData to 7 Oct 2026

Bridging MARL to SARL: An Order-Independent Multi-Agent Transformer via Latent Consensus

arXiv 2604.13472 ↗First posted 15 Apr 2026cs.LG cs.AI cs.MAcited by 2

Not on the map. Why?

Abstract (arXiv)

Cooperative multi-agent reinforcement learning (MARL) is widely used to address large joint observation and action spaces by decomposing a centralized control problem into multiple interacting agents. However, such decomposition often introduces additional challenges, including non-stationarity, unstable training, weak coordination, and limited theoretical guarantees. In this paper, we propose the Consensus Multi-Agent Transformer (CMAT), a centralized framework that bridges cooperative MARL to a hierarchical single-agent reinforcement learning (SARL) formulation. CMAT treats all agents as a unified entity and employs a Transformer encoder to process the large joint observation space. To handle the extensive joint action space, we introduce a hierarchical decision-making mechanism in which a Transformer decoder autoregressively generates a high-level consensus vector, simulating the process by which agents reach agreement on their strategies in latent space. Conditioned on this consensus, all agents generate their actions simultaneously, enabling order-independent joint decision making and avoiding the sensitivity to action-generation order in conventional Multi-Agent Transformers (MAT). This factorization allows the joint policy to be optimized using single-agent PPO while preserving expressive coordination through the latent consensus. To evaluate the proposed method, we conduct experiments on benchmark tasks from StarCraft II, Multi-Agent MuJoCo, and Google Research Football. The results show that CMAT achieves superior performance over recent centralized solutions, sequential MARL methods, and conventional MARL baselines. The code for this paper is available at:https://github.com/RS2002/CMAT .

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 61 of its 90 references that are in the map. First posted on arXiv 15 Apr 2026.

Builds on

61 of 90 references.

References · 61

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  5. 2017Proximal Policy Optimization Algorithmscited by 14,441
  6. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 1,916
  7. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
  8. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,528
  9. 2017Deep Setscited by 1,394
  10. 2021The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Gamescited by 903
  11. 2022A Generalist Agentcited by 802
  12. 2017Counterfactual Multi-Agent Policy Gradientscited by 733
38 more
  1. 2021Offline Reinforcement Learning as One Big Sequence Modeling Problemcited by 636
  2. 2019Deep Equilibrium Modelscited by 603
  3. 2019The StarCraft Multi-Agent Challengecited by 589
  4. 2020Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 475
  5. 2015Deep Recurrent Q-Learning for Partially Observable MDPscited by 457
  6. 2019QTRAN: Learning to Factorize with Transformation for Cooperative Multi-Agent Reinforcement Learningcited by 372
  7. 2016Learning Multiagent Communication with Backpropagationcited by 355
  8. 2019Stabilizing Transformers for Reinforcement Learningcited by 262
  9. 2020On the Global Convergence Rates of Softmax Policy Gradient Methodscited by 258
  10. 2019Google Research Football: A Novel Reinforcement Learning Environmentcited by 257
  11. 2022Pre-Trained Language Models for Interactive Decision-Makingcited by 220
  12. 2022Multi-Game Decision Transformerscited by 212
  13. 2018Mean Field Multi-Agent Reinforcement Learningcited by 202
  14. 2018TarMAC: Targeted Multi-Agent Communicationcited by 201
  15. 2022Online Decision Transformercited by 194
  16. 2021Trust Region Policy Optimisation in Multi-Agent Reinforcement Learningcited by 180
  17. 2020The Lipschitz Constant of Self-Attentioncited by 171
  18. 2022Multi-Agent Reinforcement Learning is a Sequence Modeling Problemcited by 153
  19. 2019A Review of Cooperative Multi-Agent Deep Reinforcement Learningcited by 147
  20. 2022TransDreamer: Reinforcement Learning with Transformer World Modelscited by 143
  21. 2022Prompting Decision Transformer for Few-Shot Policy Generalizationcited by 143
  22. 2022Q-learning Decision Transformer: Leveraging Dynamic Programming for Conditional Sequence Modelling in Offline RLcited by 103
  23. 2018Iterative Amortized Inferencecited by 93
  24. 2019On the Limitations of Representing Functions on Setscited by 89
  25. 2022Can Wikipedia Help Offline Reinforcement Learning?cited by 76
  26. 2023Heterogeneous-Agent Reinforcement Learningcited by 67
  27. 2020Value-Decomposition Multi-Agent Actor-Criticscited by 47
  28. 2024Learning Multi-Agent Communication from Graph Modeling Perspectivecited by 47
  29. 2022Masked Autoencoding for Scalable and Generalizable Decision Makingcited by 44
  30. 2023Masked Trajectory Models for Prediction, Representation, and Controlcited by 40
  31. 2021UPDeT: Universal Multi-agent Reinforcement Learning via Policy Decoupling with Transformerscited by 39
  32. 2021Shapley Counterfactual Credits for Multi-Agent Reinforcement Learningcited by 37
  33. 2023A Survey of Progress on Cooperative Multi-agent Reinforcement Learning in Open Environmentcited by 37
  34. 2022On Transforming Reinforcement Learning by Transformer: The Development Trajectorycited by 35
  35. 2025A Comprehensive Survey on Multi-Agent Cooperative Decision-Making: Scenarios, Approaches, Challenges and Perspectivescited by 27
  36. 2018Modelling the Dynamic Joint Policy of Teammates with Attention Multi-agent DDPGcited by 20
  37. 2020Convergence Proof for Actor-Critic Methods Applied to PPO and RUDDERcited by 19
  38. 2023Maximum Entropy Heterogeneous-Agent Reinforcement Learningcited by 19

50 most cited of 61 All 61 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Action-Factored Multi-Agent Reinforcement Learning for Scalable Quantum Device Tuningcited by 0
  2. 2026Aggregate in the Advantage, Not the Ratio: A Canonical-Form Analysis of Cooperative Multi-Agent Policy Optimizationcited by 0