mlsanityData to 14 Oct 2026

Graph Decision Transformer

arXiv 2303.03747 ↗First posted 7 Mar 2023cs.LG cs.AIcited by 20

On the map: Offline RL and imitation learning from data → Decision transformers and sequence modeling

Abstract (arXiv)

Offline reinforcement learning (RL) is a challenging task, whose objective is to learn policies from static trajectory data without interacting with the environment. Recently, offline RL has been viewed as a sequence modeling problem, where an agent generates a sequence of subsequent actions based on a set of static transition experiences. However, existing approaches that use transformers to attend to all tokens naively can overlook the dependencies between different tokens and limit long-term dependency learning. In this paper, we propose the Graph Decision Transformer (GDT), a novel offline RL approach that models the input sequence into a causal graph to capture potential dependencies between fundamentally different concepts and facilitate temporal and causal relationship learning. GDT uses a graph transformer to process the graph inputs with relation-enhanced mechanisms, and an optional sequence transformer to handle fine-grained spatial information in visual tasks. Our experiments show that GDT matches or surpasses the performance of state-of-the-art offline RL methods on image-based Atari and OpenAI Gym.

Key facts. Cited by 20 papers in the map (data through 14 Oct 2026). Placed in Decision transformers and sequence modeling, one of 13 directions in Offline RL and imitation learning from data. Builds on 41 of its 54 references that are in the map. First posted on arXiv 7 Mar 2023.

Builds on

41 of 54 references.

Closest ancestors · 6

Same direction, newest first

  1. 2022Q-learning Decision Transformer: Leveraging Dynamic Programming for Conditional Sequence Modelling in Offline RL8 Sep
  2. 2022Bootstrapped Transformer for Offline Reinforcement Learning17 Jun
  3. 2022Online Decision Transformer11 Feb
  4. 2021Generalized Decision Transformer for Offline Hindsight Information Matching19 Nov
  5. 2021Offline Reinforcement Learning as One Big Sequence Modeling Problem3 Jun
  6. 2021Decision Transformer: Reinforcement Learning via Sequence Modeling2 Jun

Other references · 35

  1. 2017Attention Is All You Needcited by 60,361
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  4. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,920
  5. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,809
  6. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,427
  7. 2018Relational inductive biases, deep learning, and graph networkscited by 1,360
  8. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,241
  9. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,240
  10. 2020Mastering Atari with Discrete World Modelscited by 1,007
  11. 2021A Minimalist Approach to Offline Reinforcement Learningcited by 899
  12. 2019Stabilizing Off-Policy Q-Learning via Bootstrapping Error Reductioncited by 840
23 more
  1. 2017Rainbow: Combining Improvements in Deep Reinforcement Learningcited by 786
  2. 2019Advantage-Weighted Regression: Simple and Scalable Off-Policy Reinforcement Learningcited by 782
  3. 2019Behavior Regularized Offline Reinforcement Learningcited by 662
  4. 2020MOPO: Model-based Offline Policy Optimizationcited by 636
  5. 2020MOReL : Model-Based Offline Reinforcement Learningcited by 590
  6. 2017Distributional Reinforcement Learning with Quantile Regressioncited by 474
  7. 2018Adversarial Attack on Graph Structured Datacited by 301
  8. 2021Offline Reinforcement Learning with Fisher Divergence Critic Regularizationcited by 238
  9. 2020Keep Doing What Worked: Behavioral Modelling Priors for Offline Reinforcement Learningcited by 215
  10. 2020Model-Based Offline Planningcited by 129
  11. 2020One Policy to Control Them All: Shared Modular Policies for Agent-Agnostic Controlcited by 121
  12. 2019Training Agents using Upside-Down Reinforcement Learningcited by 107
  13. 2019Reward-Conditioned Policiescited by 102
  14. 2019Deep Coordination Graphscited by 98
  15. 2019Graph Transformer for Graph-to-Sequence Learningcited by 82
  16. 2019Auto-GNN: Neural Architecture Search of Graph Neural Networkscited by 74
  17. 2020Rewriting History with Inverse RL: Hindsight Inference for Policy Improvementcited by 60
  18. 2020Generalized Hindsight for Reinforcement Learningcited by 51
  19. 2022Challenges and Opportunities in Deep Reinforcement Learning with Graph Neural Networks: A Comprehensive review of Algorithms and Applicationscited by 30
  20. 2021StARformer: Transformer with State-Action-Reward Representations for Visual Reinforcement Learningcited by 27
  21. 2022Addressing Optimism Bias in Sequence Modeling for Reinforcement Learningcited by 22
  22. 2023SaFormer: A Conditional Sequence Modeling Approach to Offline Safe Reinforcement Learningcited by 16
  23. 2022Curriculum-based Asymmetric Multi-task Reinforcement Learningcited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 13; 2025: 4; 2026: 1.

20 papers.

Sort:

By area:

  1. 2024Learning Multi-Agent Communication from Graph Modeling Perspectivecited by 48
  2. 2023Crossway Diffusion: Improving Diffusion-based Visuomotor Policy via Self-supervised Learningcited by 40
  3. 2024Q-value Regularized Transformer for Offline Reinforcement Learningcited by 29
  4. 2024Deep Generative Models for Offline Policy Learning: Tutorial, Survey, and Perspectives on Future Directionscited by 20
  5. 2023Prompt-Tuning Decision Transformer with Preference Rankingcited by 19
  6. 2024Decision Mamba: A Multi-Grained State Space Model with Self-Evolution Regularization for Offline RLcited by 15
  7. 2024HarmoDT: Harmony Multi-Task Decision Transformer for Offline Reinforcement Learningcited by 14
  8. 2024Gradformer: Graph Transformer with Exponential Decaycited by 11
  9. 2024Temporal Logic Specification-Conditioned Decision Transformer for Offline Safe Reinforcement Learningcited by 10
  10. 2024Is Mamba Compatible with Trajectory Optimization in Offline Reinforcement Learning?cited by 7
  11. 2024Prompt Tuning with Diffusion for Few-Shot Pre-trained Policy Generalizationcited by 6
  12. 2025TrojanTO: Action-Level Backdoor Attacks against Trajectory Optimization Modelscited by 3
8 more
  1. 2025Should We Ever Prefer Decision Transformer for Offline Reinforcement Learning?cited by 2
  2. 2024Task-Aware Harmony Multi-Task Decision Transformer for Offline Reinforcement Learningcited by 1
  3. 2024Continual Task Learning through Adaptive Policy Self-Compositioncited by 1
  4. 2024Communication Learning in Multi-Agent Systems from Graph Modeling Perspectivecited by 0
  5. 2024Are Expressive Models Truly Necessary for Offline RL?cited by 0
  6. 2025Decision Flow Policy Optimizationcited by 0
  7. 2025Adjusting the Output of Decision Transformer with Action Gradientcited by 0
  8. 2026QHyer: Q-conditioned Hybrid Attention-mamba Transformer for Offline Goal-conditioned RLcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Scaling Offline Model-Based RL via Jointly-Optimized World-Action Model Pretraining
  2. 2023Multi-Objective Decision Transformers for Offline Reinforcement Learning
  3. 2022Contextual Transformer for Offline Meta Reinforcement Learning
  4. 2022Value Memory Graph: A Graph-Structured World Model for Offline Reinforcement Learning
  5. 2020Offline Reinforcement Learning from Images with Latent Space Models
  6. 2023Rethinking Decision Transformer via Hierarchical Reinforcement Learning
  7. 2022Offline Reinforcement Learning with Causal Structured World Models
  8. 2023Waypoint Transformer: Reinforcement Learning via Supervised Learning with Intermediate Targets