mlsanityData to 7 Oct 2026

Decoupling Return-to-Go for Efficient Decision Transformer

arXiv 2601.15953 ↗First posted 22 Jan 2026cs.AIcited by 1

On the map: Offline RL and imitation learning from data → Decision transformers and sequence modeling

Abstract (arXiv)

The Decision Transformer (DT) has established a powerful sequence modeling approach to offline reinforcement learning. It conditions its action predictions on Return-to-Go (RTG), using it both to distinguish trajectory quality during training and to guide action generation at inference. In this work, we identify a critical redundancy in this design: feeding the entire sequence of RTGs into the Transformer is theoretically unnecessary, as only the most recent RTG affects action prediction. We show that this redundancy can impair DT's performance through experiments. To resolve this, we propose the Decoupled DT (DDT). DDT simplifies the architecture by processing only observation and action sequences through the Transformer, using the latest RTG to guide the action prediction. This streamlined approach not only improves performance but also reduces computational cost. Our experiments show that DDT significantly outperforms DT and establishes competitive performance against state-of-the-art DT variants across multiple offline RL tasks.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Decision transformers and sequence modeling, one of 13 directions in Offline RL and imitation learning from data. Builds on 33 of its 39 references that are in the map. First posted on arXiv 22 Jan 2026.

Builds on

33 of 39 references.

Closest ancestors · 13

Same direction, newest first

  1. 2024Reinforcement Learning Gradients as Vitamin for Online Finetuning Decision Transformers31 Oct
  2. 2024Decision Mamba: A Multi-Grained State Space Model with Self-Evolution Regularization for Offline RL8 Jun
  3. 2024Decision Mamba: Reinforcement Learning via Hybrid Selective Sequence Modeling31 May
  4. 2023Critic-Guided Decision Transformer for Offline Reinforcement Learning21 Dec
  5. 2023Decision ConvFormer: Local Filtering in MetaFormer is Sufficient for Decision Making4 Oct
  6. 2023ACT: Empowering Decision Transformer with Dynamic Programming via Advantage Conditioning12 Sep
  7. 2023Waypoint Transformer: Reinforcement Learning via Supervised Learning with Intermediate Targets24 Jun
  8. 2022Dichotomy of Control: Separating What You Can Control from What You Cannot24 Oct
  9. 2022Q-learning Decision Transformer: Leveraging Dynamic Programming for Conditional Sequence Modelling in Offline RL8 Sep
  10. 2022You Can't Count on Luck: Why Decision Transformers and RvS Fail in Stochastic Environments31 May
  11. 2022Online Decision Transformer11 Feb
  12. 2021Generalized Decision Transformer for Offline Hindsight Information Matching19 Nov
1 more
  1. 2021Decision Transformer: Reinforcement Learning via Sequence Modeling2 Jun

Other references · 20

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  3. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  4. 2022Scalable Diffusion Models with Transformerscited by 5,931
  5. 2016Layer Normalizationcited by 5,579
  6. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,616
  7. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  8. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,777
  9. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,406
  10. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,245
  11. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,227
  12. 2018DeepMind Control Suitecited by 935
8 more
  1. 2021A Minimalist Approach to Offline Reinforcement Learningcited by 882
  2. 2019Behavior Regularized Offline Reinforcement Learningcited by 654
  3. 2017Modulating early visual processing by languagecited by 167
  4. 2019Reward-Conditioned Policiescited by 101
  5. 2021d3rlpy: An Offline Deep Reinforcement Learning Librarycited by 84
  6. 2022When does return-conditioned supervised learning work for offline reinforcement learning?cited by 73
  7. 2020Approximate information state for approximate planning and reinforcement learning in partially observed systemscited by 67
  8. 2023Elastic Decision Transformercited by 66

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Beyond Autoregressive RTG: Conditioning via Injection Outside Sequential Modeling in Decision Transformercited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Should We Ever Prefer Decision Transformer for Offline Reinforcement Learning?
  2. 2026Online Finetuning Decision Transformers with Pure RL Gradients
  3. 2024Enhancing Decision Transformer with Diffusion-Based Trajectory Branch Generation
  4. 2023Rethinking Decision Transformer via Hierarchical Reinforcement Learning
  5. 2024Q-value Regularized Decision ConvFormer for Offline Reinforcement Learning
  6. 2025A Comparison Between Decision Transformers and Traditional Offline Reinforcement Learning Algorithms
  7. 2024Return-Aligned Decision Transformer
  8. 2025DRDT3: Diffusion-Refined Decision Test-Time Training Model