mlsanityData to 7 Oct 2026

Skill Decision Transformer

arXiv 2301.13573 ↗First posted 31 Jan 2023cs.LGcited by 7

On the map: Offline RL and imitation learning from data → Decision transformers and sequence modeling

Abstract (arXiv)

Recent work has shown that Large Language Models (LLMs) can be incredibly effective for offline reinforcement learning (RL) by representing the traditional RL problem as a sequence modelling problem (Chen et al., 2021; Janner et al., 2021). However many of these methods only optimize for high returns, and may not extract much information from a diverse dataset of trajectories. Generalized Decision Transformers (GDTs) (Furuta et al., 2021) have shown that utilizing future trajectory information, in the form of information statistics, can help extract more information from offline trajectory data. Building upon this, we propose Skill Decision Transformer (Skill DT). Skill DT draws inspiration from hindsight relabelling (Andrychowicz et al., 2017) and skill discovery methods to discover a diverse set of primitive behaviors, or skills. We show that Skill DT can not only perform offline state-marginal matching (SMM), but can discovery descriptive behaviors that can be easily sampled. Furthermore, we show that through purely reward-free optimization, Skill DT is still competitive with supervised offline RL approaches on the D4RL benchmark. The code and videos can be found on our project page: https://github.com/shyamsn97/skill-dt

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Placed in Decision transformers and sequence modeling, one of 13 directions in Offline RL and imitation learning from data. Builds on 26 of its 29 references that are in the map. First posted on arXiv 31 Jan 2023.

Builds on

26 of 29 references.

Closest ancestors · 4

Same direction, newest first

  1. 2022Online Decision Transformer11 Feb
  2. 2021Generalized Decision Transformer for Offline Hindsight Information Matching19 Nov
  3. 2021Offline Reinforcement Learning as One Big Sequence Modeling Problem3 Jun
  4. 2021Decision Transformer: Reinforcement Learning via Sequence Modeling2 Jun

Other references · 22

  1. 2017Attention Is All You Needcited by 59,932
  2. 2017Neural Discrete Representation Learningcited by 4,650
  3. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  4. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,777
  5. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,406
  6. 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,376
  7. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,245
  8. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,173
  9. 2017Hindsight Experience Replaycited by 1,096
  10. 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 734
  11. 2017Deep Reinforcement Learning: An Overviewcited by 327
  12. 2019Dynamics-Aware Unsupervised Discovery of Skillscited by 308
10 more
  1. 2016Variational Intrinsic Controlcited by 262
  2. 2019Efficient Exploration via State Marginal Matchingcited by 186
  3. 2020OPAL: Offline Primitive Discovery for Accelerating Offline Reinforcement Learningcited by 164
  4. 2020Explore, Discover and Learn: Unsupervised Discovery of State-Covering Skillscited by 120
  5. 2021Vector Quantized Models for Planningcited by 47
  6. 2021Learning more skills through optimistic explorationcited by 37
  7. 2017Hindsight policy gradientscited by 36
  8. 2019Learning to Reach Goals via Iterated Supervised Learningcited by 26
  9. 2022Goal-Conditioned Generators of Deep Policiescited by 9
  10. 2022Robust Vector Quantized-Variational Autoencodercited by 4

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 5; 2025: 1.

7 papers.

  1. 2023Generalization to New Sequential Decision Making Tasks with In-Context Learningcited by 39
  2. 2024Deep Generative Models for Offline Policy Learning: Tutorial, Survey, and Perspectives on Future Directionscited by 20
  3. 2024Reinforcement Learning Gradients as Vitamin for Online Finetuning Decision Transformerscited by 8
  4. 2024Few-Shot Task Learning through Inverse Generative Modelingcited by 4
  5. 2024Navigation with QPHIL: Quantizing Planner for Hierarchical Implicit Q-Learningcited by 2
  6. 2024Human-like Bots for Tactical Shooters Using Compute-Efficient Sensorscited by 1
  7. 2025Robust Adversarial Reinforcement Learning in Stochastic Games via Sequence Modelingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Pre-trained Language Models Improve the Few-shot Prompt Ability of Decision Transformer
  2. 2025A Comparison Between Decision Transformers and Traditional Offline Reinforcement Learning Algorithms
  3. 2026Online Finetuning Decision Transformers with Pure RL Gradients
  4. 2023When should we prefer Decision Transformers for Offline Reinforcement Learning?
  5. 2022Q-learning Decision Transformer: Leveraging Dynamic Programming for Conditional Sequence Modelling in Offline RL
  6. 2025Should We Ever Prefer Decision Transformer for Offline Reinforcement Learning?
  7. 2023Multi-Objective Decision Transformers for Offline Reinforcement Learning
  8. 2025Offline Reinforcement Learning with Discrete Diffusion Skills