mlsanityData to 13 Oct 2026

Unified token representations for sequential decision models

arXiv 2510.21448 ↗First posted 24 Oct 2025cs.LGcited by 0

On the map: Offline RL and imitation learning from data → Decision transformers and sequence modeling

also in 1 more
Abstract (arXiv)

Transformers have demonstrated strong potential in offline reinforcement learning (RL) by modeling trajectories as sequences of return-to-go, states, and actions. However, existing approaches such as the Decision Transformer(DT) and its variants suffer from redundant tokenization and quadratic attention complexity, limiting their scalability in real-time or resource-constrained settings. To address this, we propose a Unified Token Representation (UTR) that merges return-to-go, state, and action into a single token, substantially reducing sequence length and model complexity. Theoretical analysis shows that UTR leads to a tighter Rademacher complexity bound, suggesting improved generalization. We further develop two variants: UDT and UDC, built upon transformer and gated CNN backbones, respectively. Both achieve comparable or superior performance to state-of-the-art methods with markedly lower computation. These findings demonstrate that UTR generalizes well across architectures and may provide an efficient foundation for scalable control in future large decision models.

Key facts. No citing papers in the map yet (data through 13 Oct 2026). Placed in Decision transformers and sequence modeling, one of 13 directions in Offline RL and imitation learning from data. Builds on 14 of its 17 references that are in the map. First posted on arXiv 24 Oct 2025.

Builds on

14 of 17 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024Decision Mamba: Reinforcement Learning via Sequence Modeling with Selective State Spaces29 Mar
  2. 2023Decision ConvFormer: Local Filtering in MetaFormer is Sufficient for Decision Making4 Oct
  3. 2021Decision Transformer: Reinforcement Learning via Sequence Modeling2 Jun

Other references · 11

  1. 2017Attention Is All You Needcited by 60,106
  2. 2020Language Models are Few-Shot Learnerscited by 32,320
  3. 2022A ConvNet for the 2020scited by 3,563
  4. 2016WaveNet: A Generative Model for Raw Audiocited by 1,717
  5. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,415
  6. 2023ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoderscited by 705
  7. 2016Language Modeling with Gated Convolutional Networkscited by 694
  8. 2021MetaFormer Is Actually What You Need for Visioncited by 524
  9. 2022A Survey on Offline Reinforcement Learning: Taxonomy, Review, and Open Problemscited by 228
  10. 2024MambaOut: Do We Really Need Mamba for Vision?cited by 143
  11. 2022Global Context Vision Transformerscited by 98

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Planning Transformer: Long-Horizon Offline Reinforcement Learning with Planning Tokens
  2. 2024Pre-trained Language Models Improve the Few-shot Prompt Ability of Decision Transformer
  3. 2023Transformers in Reinforcement Learning: A Survey
  4. 2022UniMASK: Unified Inference in Sequential Decision Problems
  5. 2023Rethinking Decision Transformer via Hierarchical Reinforcement Learning
  6. 2026Beyond Autoregressive RTG: Conditioning via Injection Outside Sequential Modeling in Decision Transformer
  7. 2026Not all tokens are needed(NAT): token efficient reinforcement learning
  8. 2026Reinforcement Learning with Promising Tokens for Large Language Models