mlsanityData to 14 Oct 2026

Trust Guided Decision Transformer

arXiv 2609.31586 ↗First posted 25 Sep 2026cs.LGcited by 0

On the map: Offline RL and imitation learning from data → Decision transformers and sequence modeling

Abstract (arXiv)

Decision Transformer performance degrades on long rollouts because the conditioning context drifts out of the training distribution. We show that this drift is visible through the model's own next state prediction error, which rises during rollout and stays elevated, giving a direct signal of when context has become unreliable. We introduce Trust Guided Decision Transformer (TGDT), which selects context before applying value guidance. At each step, TGDT evaluates several recent context suffixes using rolling next state prediction error, calibrated against held out offline data via split conformal prediction. It keeps only suffixes whose error stays within the calibrated threshold, then uses a frozen critic to choose the highest value action among the trusted suffixes. This reverses the order used by value only elastic selection, where the critic may choose an action generated from a context the model itself has flagged as unreliable. Experiments on D4RL navigation and locomotion tasks show that state prediction, critic guidance, and hard context reset each solve only part of the problem. TGDT reduces persistent high error runs and improves return over vanilla Decision Transformer, reset based context control, and value only context selection.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Decision transformers and sequence modeling, one of 13 directions in Offline RL and imitation learning from data. Builds on 25 of its 30 references that are in the map. First posted on arXiv 25 Sep 2026.

Builds on

25 of 30 references.

Closest ancestors · 6

Same direction, newest first

  1. 2023Critic-Guided Decision Transformer for Offline Reinforcement Learning21 Dec
  2. 2023Decision ConvFormer: Local Filtering in MetaFormer is Sufficient for Decision Making4 Oct
  3. 2023ACT: Empowering Decision Transformer with Dynamic Programming via Advantage Conditioning12 Sep
  4. 2022Q-learning Decision Transformer: Leveraging Dynamic Programming for Conditional Sequence Modelling in Offline RL8 Sep
  5. 2021Offline Reinforcement Learning as One Big Sequence Modeling Problem3 Jun
  6. 2021Decision Transformer: Reinforcement Learning via Sequence Modeling2 Jun

Other references · 19

  1. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,809
  2. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,427
  3. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,293
  4. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,241
  5. 2021A Minimalist Approach to Offline Reinforcement Learningcited by 899
  6. 2019Stabilizing Off-Policy Q-Learning via Bootstrapping Error Reductioncited by 840
  7. 2019Advantage-Weighted Regression: Simple and Scalable Off-Policy Reinforcement Learningcited by 782
  8. 2020MOPO: Model-based Offline Policy Optimizationcited by 636
  9. 2020MOReL : Model-Based Offline Reinforcement Learningcited by 590
  10. 2022Diffusion Policies as an Expressive Policy Class for Offline Reinforcement Learningcited by 562
  11. 2019Conformal Prediction Under Covariate Shiftcited by 519
  12. 2022Is Conditional Generative Modeling all you need for Decision-Making?cited by 516
7 more
  1. 2021Adaptive Conformal Inference Under Distribution Shiftcited by 375
  2. 2021COMBO: Conservative Offline Model-Based Policy Optimizationcited by 370
  3. 2022Conformal prediction beyond exchangeabilitycited by 355
  4. 2021Uncertainty-Based Offline Reinforcement Learning with Diversified Q-Ensemblecited by 302
  5. 2021Offline RL Without Off-Policy Evaluationcited by 151
  6. 2022When does return-conditioned supervised learning work for offline reinforcement learning?cited by 76
  7. 2023Elastic Decision Transformercited by 67

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Decoupling Return-to-Go for Efficient Decision Transformer
  2. 2026Beyond Autoregressive RTG: Conditioning via Injection Outside Sequential Modeling in Decision Transformer
  3. 2023Rethinking Decision Transformer via Hierarchical Reinforcement Learning
  4. 2024Predictive Coding for Decision Transformer
  5. 2025Learning to Trust Bellman Updates: Selective State-Adaptive Regularization for Offline RL
  6. 2026In-Context Reinforcement Learning From Suboptimal Historical Data
  7. 2025DRDT3: Diffusion-Refined Decision Test-Time Training Model
  8. 2026Beyond Uniform Token-Level Trust Region in LLM Reinforcement Learning