mlsanityData to 7 Oct 2026

Stackelberg Decision Transformer for Asynchronous Action Coordination in Multi-Agent Systems

arXiv 2305.07856 ↗First posted 13 May 2023cs.MA cs.AIcited by 1

On the map: Offline RL and imitation learning from data → Offline multi-agent RL

Abstract (arXiv)

Asynchronous action coordination presents a pervasive challenge in Multi-Agent Systems (MAS), which can be represented as a Stackelberg game (SG). However, the scalability of existing Multi-Agent Reinforcement Learning (MARL) methods based on SG is severely constrained by network structures or environmental limitations. To address this issue, we propose the Stackelberg Decision Transformer (STEER), a heuristic approach that resolves the difficulties of hierarchical coordination among agents. STEER efficiently manages decision-making processes in both spatial and temporal contexts by incorporating the hierarchical decision structure of SG, the modeling capability of autoregressive sequence models, and the exploratory learning methodology of MARL. Our research contributes to the development of an effective and adaptable asynchronous action coordination method that can be widely applied to various task types and environmental configurations in MAS. Experimental results demonstrate that our method can converge to Stackelberg equilibrium solutions and outperforms other existing methods in complex scenarios.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Offline multi-agent RL, one of 13 directions in Offline RL and imitation learning from data. Builds on 26 of its 36 references that are in the map. First posted on arXiv 13 May 2023.

Builds on

26 of 36 references.

References · 26

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2017Proximal Policy Optimization Algorithmscited by 14,441
  5. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  6. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,528
  7. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,519
  8. 2021The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Gamescited by 903
  9. 2017An Overview of Multi-Task Learning in Deep Neural Networkscited by 791
  10. 2018QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 697
  11. 2021Offline Reinforcement Learning as One Big Sequence Modeling Problemcited by 636
  12. 2021Transformer in Transformercited by 632
14 more
  1. 2020Time Series Forecasting With Deep Learning: A Surveycited by 457
  2. 2019Google Research Football: A Novel Reinforcement Learning Environmentcited by 257
  3. 2021Trust Region Policy Optimisation in Multi-Agent Reinforcement Learningcited by 180
  4. 2020FACMAC: Factored Multi-Agent Centralised Policy Gradientscited by 178
  5. 2022Multi-Agent Reinforcement Learning is a Sequence Modeling Problemcited by 153
  6. 2021Investigating Bi-Level Optimization for Learning and Vision from a Unified Perspective: A Survey and Beyondcited by 151
  7. 2017A Review on Bilevel Optimization: From Classical to Evolutionary Approaches and Applicationscited by 126
  8. 2021UPDeT: Universal Multi-agent Reinforcement Learning via Policy Decoupling with Transformerscited by 39
  9. 2019Bi-level Actor-Critic for Multi-agent Coordinationcited by 38
  10. 2023Order Matters: Agent-by-agent Policy Optimizationcited by 24
  11. 2022Transformer in Transformer as Backbone for Deep Reinforcement Learningcited by 6
  12. 2023Inducing Stackelberg Equilibrium through Spatio-Temporal Sequential Decision-Making in Multi-Agent Reinforcement Learningcited by 5
  13. 2020Algorithms in Multi-Agent Systems: A Holistic Perspective from Reinforcement Learning and Game Theorycited by 2
  14. 2023Dual Self-Awareness Value Decomposition Framework without Individual Global Max for Cooperative Multi-Agent Reinforcement Learningcited by 1

Built on by

Cited by year

Citing papers per year: 2023: 1.

1 paper.

  1. 2023Controlling Large Language Model-based Agents for Large-Scale Decision-Making: An Actor-Critic Approachcited by 38

Similar, not cited

Close in wording, no citation either way

  1. 2026TSC: Topology-Conditioned Stackelberg Coordination for Multi-Agent Reinforcement Learning in Interactive Driving
  2. 2021Offline Pre-trained Multi-Agent Decision Transformer: One Big Sequence Model Tackles All SMAC Tasks
  3. 2023A Model-Based Solution to the Offline Multi-Agent Reinforcement Learning Coordination Problem
  4. 2024Decentralized Transformers with Centralized Aggregation are Sample-Efficient Multi-Agent World Models
  5. 2025Transformer World Model for Sample Efficient Multi-Agent Reinforcement Learning
  6. 2025Offline Multi-agent Reinforcement Learning via Sequential Score Decomposition
  7. 2025Oryx: a Scalable Sequence Model for Many-Agent Coordination in Offline MARL
  8. 2026CODA: Coordination via On-Policy Diffusion for Multi-Agent Offline Reinforcement Learning