mlsanityData to 14 Oct 2026

Return-Aligned Decision Transformer

arXiv 2402.03923 ↗First posted 6 Feb 2024cs.LGcited by 3

On the map: Offline RL and imitation learning from data → Decision transformers and sequence modeling

Abstract (arXiv)

Traditional approaches in offline reinforcement learning aim to learn the optimal policy that maximizes the cumulative reward, also known as return. It is increasingly important to adjust the performance of AI agents to meet human requirements, for example, in applications like video games and education tools. Decision Transformer (DT) optimizes a policy that generates actions conditioned on the target return through supervised learning and includes a mechanism to control the agent's performance using the target return. However, the action generation is hardly influenced by the target return because DT's self-attention allocates scarce attention scores to the return tokens. In this paper, we propose Return-Aligned Decision Transformer (RADT), designed to more effectively align the actual return with the target return. RADT leverages features extracted by paying attention solely to the return, enabling action generation to consistently depend on the target return. Extensive experiments show that RADT significantly reduces the discrepancies between the actual return and the target return compared to DT-based methods. Our code is available at https://github.com/CyberAgentAILab/radt

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Decision transformers and sequence modeling, one of 13 directions in Offline RL and imitation learning from data. Builds on 38 of its 52 references that are in the map. First posted on arXiv 6 Feb 2024.

Builds on

38 of 52 references.

Closest ancestors · 8

Same direction, newest first

  1. 2023Critic-Guided Decision Transformer for Offline Reinforcement Learning21 Dec
  2. 2023Decision ConvFormer: Local Filtering in MetaFormer is Sufficient for Decision Making4 Oct
  3. 2023ACT: Empowering Decision Transformer with Dynamic Programming via Advantage Conditioning12 Sep
  4. 2023Decision S4: Efficient Sequence-Based RL via State Spaces Layers8 Jun
  5. 2022Q-learning Decision Transformer: Leveraging Dynamic Programming for Conditional Sequence Modelling in Offline RL8 Sep
  6. 2022Online Decision Transformer11 Feb
  7. 2021Offline Reinforcement Learning as One Big Sequence Modeling Problem3 Jun
  8. 2021Decision Transformer: Reinforcement Learning via Sequence Modeling2 Jun

Other references · 30

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
  4. 2017CARLA: An Open Urban Driving Simulatorcited by 2,449
  5. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,920
  6. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,809
  7. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,427
  8. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,293
  9. 2020Mastering Atari with Discrete World Modelscited by 1,007
  10. 2021A Minimalist Approach to Offline Reinforcement Learningcited by 899
  11. 2021COMBO: Conservative Offline Model-Based Policy Optimizationcited by 370
  12. 2020Is Pessimism Provably Efficient for Offline RL?cited by 364
18 more
  1. 2023OmniControl: Control Any Joint at Any Time for Human Motion Generationcited by 238
  2. 2021RvS: What is Essential for Offline RL via Supervised Learning?cited by 185
  3. 2023Trace and Pace: Controllable Pedestrian Animation via Guided Trajectory Diffusioncited by 138
  4. 2019Reinforcement Learning Upside Down: Don't Predict Rewards -- Just Map Them to Actionscited by 125
  5. 2019Training Agents using Upside-Down Reinforcement Learningcited by 107
  6. 2023Human Motion Generation: A Surveycited by 96
  7. 2023Elastic Decision Transformercited by 67
  8. 2022A Policy-Guided Imitation Approach for Offline Reinforcement Learningcited by 59
  9. 2022GRIT: Faster and Better Image captioning Transformer Using Dual Visual Featurescited by 56
  10. 2021Augmenting Automated Game Testing with Deep Reinforcement Learningcited by 38
  11. 2024DiffStitch: Boosting Offline Reinforcement Learning with Diffusion-based Trajectory Stitchingcited by 36
  12. 2023Emergent Agentic Transformer from Chain of Hindsight Experiencecited by 33
  13. 2021StARformer: Transformer with State-Action-Reward Representations for Visual Reinforcement Learningcited by 27
  14. 2021Reinforcement Learning for Education: Opportunities and Challengescited by 25
  15. 2019Automated Video Game Testing Using Synthetic and Human-Like Agentscited by 22
  16. 2021An Overview of Agent-based Traffic Simulatorscited by 11
  17. 2023H-GAP: Humanoid Control with a Generalist Plannercited by 11
  18. 2023RaidEnv: Exploring New Challenges in Automated Content Balancing for Boss Raid Gamescited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 1; 2026: 1.

3 papers.

  1. 2024In-Dataset Trajectory Return Regularization for Offline Preference-based Reinforcement Learningcited by 3
  2. 2025Target Return Optimizer for Multi-Game Decision Transformercited by 0
  3. 2026Return-to-Go Is More Than a Number: Q-Guided Alignment for Return-Conditioned Supervised Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Return Augmented Decision Transformer for Off-Dynamics Reinforcement Learning
  2. 2025Prompting Decision Transformers for Zero-Shot Reach-Avoid Policies
  3. 2026Decoupling Return-to-Go for Efficient Decision Transformer
  4. 2025Adjusting the Output of Decision Transformer with Action Gradient
  5. 2024Predictive Coding for Decision Transformer
  6. 2024Reinforcement Learning Gradients as Vitamin for Online Finetuning Decision Transformers
  7. 2025Should We Ever Prefer Decision Transformer for Offline Reinforcement Learning?
  8. 2022When does return-conditioned supervised learning work for offline reinforcement learning?