mlsanityData to 14 Oct 2026

Toward Explainable Offline RL: Analyzing Representations in Intrinsically Motivated Decision Transformers

arXiv 2506.13958 ↗First posted 16 Jun 2025cs.LG cs.AIcited by 0

On the map: Offline RL and imitation learning from data → Decision transformers and sequence modeling

Abstract (arXiv)

Elastic Decision Transformers (EDTs) have proved to be particularly successful in offline reinforcement learning, offering a flexible framework that unifies sequence modeling with decision-making under uncertainty. Recent research has shown that incorporating intrinsic motivation mechanisms into EDTs improves performance across exploration tasks, yet the representational mechanisms underlying these improvements remain unexplored. In this paper, we introduce a systematic post-hoc explainability framework to analyze how intrinsic motivation shapes learned embeddings in EDTs. Through statistical analysis of embedding properties (including covariance structure, vector magnitudes, and orthogonality), we reveal that different intrinsic motivation variants create fundamentally different representational structures. Our analysis demonstrates environment-specific correlation patterns between embedding metrics and performance that explain why intrinsic motivation improves policy learning. These findings show that intrinsic motivation operates beyond simple exploration bonuses, acting as a representational prior that shapes embedding geometry in biologically plausible ways, creating environment-specific organizational structures that facilitate better decision-making.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Decision transformers and sequence modeling, one of 13 directions in Offline RL and imitation learning from data. Builds on 14 of its 26 references that are in the map. First posted on arXiv 16 Jun 2025.

Builds on

14 of 26 references.

Closest ancestors · 1

Same direction, newest first

  1. 2021Decision Transformer: Reinforcement Learning via Sequence Modeling2 Jun

Other references · 13

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,920
  3. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,809
  4. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,427
  5. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,194
  6. 2018Exploration by Random Network Distillationcited by 993
  7. 2016Unifying Count-Based Exploration and Intrinsic Motivationcited by 639
  8. 2020MOReL : Model-Based Offline Reinforcement Learningcited by 590
  9. 2022A Survey on Offline Reinforcement Learning: Taxonomy, Review, and Open Problemscited by 231
  10. 2023Elastic Decision Transformercited by 67
  11. 2022Redeeming Intrinsic Rewards via Constrained Optimizationcited by 24
  12. 2022Unsupervised Representation Learning in Deep Reinforcement Learning: A Reviewcited by 18
1 more
  1. 2022Foundation Models for Semantic Novelty in Reinforcement Learningcited by 9

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Fostering Intrinsic Motivation in Reinforcement Learning with Pretrained Foundation Models
  2. 2026Momentum for Reasoning: Dense Intrinsic Signals in Policy Optimization
  3. 2026ETR: Outcome-Guided Elastic Trust Regions for Policy Optimization
  4. 2023Intrinsic Motivation in Model-based Reinforcement Learning: A Brief Review
  5. 2024In-Context Decision Transformer: Reinforcement Learning via Hierarchical Chain-of-Thought
  6. 2024PreND: Enhancing Intrinsic Motivation in Reinforcement Learning through Pre-trained Network Distillation
  7. 2025From Memories to Maps: Mechanisms of In-Context Reinforcement Learning in Transformers
  8. 2025Explainable Reinforcement Learning via Temporal Policy Decomposition