mlsanityData to 7 Oct 2026

DODT: Enhanced Online Decision Transformer Learning through Dreamer's Actor-Critic Trajectory Forecasting

arXiv 2410.11359 ↗First posted 15 Oct 2024cs.LG cs.RO stat.MLcited by 1

On the map: Offline RL and imitation learning from data → Decision transformers and sequence modeling

Abstract (arXiv)

Advancements in reinforcement learning have led to the development of sophisticated models capable of learning complex decision-making tasks. However, efficiently integrating world models with decision transformers remains a challenge. In this paper, we introduce a novel approach that combines the Dreamer algorithm's ability to generate anticipatory trajectories with the adaptive learning strengths of the Online Decision Transformer. Our methodology enables parallel training where Dreamer-produced trajectories enhance the contextual decision-making of the transformer, creating a bidirectional enhancement loop. We empirically demonstrate the efficacy of our approach on a suite of challenging benchmarks, achieving notable improvements in sample efficiency and reward maximization over existing methods. Our results indicate that the proposed integrated framework not only accelerates learning but also showcases robustness in diverse and dynamic scenarios, marking a significant step forward in model-based reinforcement learning.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Decision transformers and sequence modeling, one of 13 directions in Offline RL and imitation learning from data. Builds on 9 of its 18 references that are in the map. First posted on arXiv 15 Oct 2024.

Builds on

9 of 18 references.

Closest ancestors · 2

Same direction, newest first

  1. 2021Offline Reinforcement Learning as One Big Sequence Modeling Problem3 Jun
  2. 2021Decision Transformer: Reinforcement Learning via Sequence Modeling2 Jun

Other references · 7

  1. 2017Attention Is All You Needcited by 59,932
  2. 2022TransDreamer: Reinforcement Learning with Transformer World Modelscited by 143
  3. 2023A Survey on Transformers in Reinforcement Learningcited by 50
  4. 2021Latent Imagination Facilitates Zero-Shot Transfer in Autonomous Racingcited by 22
  5. 2024Latent Plan Transformer for Trajectory Abstraction: Planning as Latent Space Inferencecited by 14
  6. 2022Latent State Marginalization as a Low-cost Approach for Improving Explorationcited by 13
  7. 2023Decision Stacks: Flexible Reinforcement Learning via Modular Generative Modelscited by 8

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025Learning to Rank Chain-of-Thought: Using a Small Modelcited by 5

Similar, not cited

Close in wording, no citation either way

  1. 2024Goal-Conditioned Decision Transformer for Multi-Goal Offline Reinforcement Learning
  2. 2024Reinforcement Learning Gradients as Vitamin for Online Finetuning Decision Transformers
  3. 2023ACT: Empowering Decision Transformer with Dynamic Programming via Advantage Conditioning
  4. 2024Enhancing Decision Transformer with Diffusion-Based Trajectory Branch Generation
  5. 2023Critic-Guided Decision Transformer for Offline Reinforcement Learning
  6. 2025Adjusting the Output of Decision Transformer with Action Gradient
  7. 2026In-Context Reinforcement Learning From Suboptimal Historical Data
  8. 2024In-Context Decision Transformer: Reinforcement Learning via Hierarchical Chain-of-Thought