# Adaptive Transformers in RL

arXiv: https://arxiv.org/abs/2004.03761 · HTML: https://mlsanity.com/paper/2004.03761/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Apr 2020.

## Built on by (most cited first, 13 of 13)

- [Offline Reinforcement Learning as One Big Sequence Modeling Problem](https://mlsanity.com/paper/2106.02039/) (2021)
- [Pathologies in priors and inference for Bayesian transformers](https://mlsanity.com/paper/2110.04020/) (2021)
- [Explain My Surprise: Learning Efficient Long-Term Memory by Predicting Uncertain Outcomes](https://mlsanity.com/paper/2207.13649/) (2022)
- [Lyapunov-based uncertainty-aware safe reinforcement learning](https://mlsanity.com/paper/2107.13944/) (2021)
- [Provably Improved Context-Based Offline Meta-RL with Attention and Contrastive Learning](https://mlsanity.com/paper/2102.10774/) (2021)
- [Learning Computational Efficient Bots with Costly Features](https://mlsanity.com/paper/2308.09629/) (2023)
- [Navigation with QPHIL: Quantizing Planner for Hierarchical Implicit Q-Learning](https://mlsanity.com/paper/2411.07760/) (2024)
- [Deep Generative Models for Decision-Making and Control](https://mlsanity.com/paper/2306.08810/) (2023)
- [Sample-based Dynamic Hierarchical Transformer with Layer and Head Flexibility via Contextual Bandit](https://mlsanity.com/paper/2312.03038/) (2023)
- [Neural architecture impact on identifying temporally extended Reinforcement Learning tasks](https://mlsanity.com/paper/2310.03161/) (2023)
- [Diverse Transformer Decoding for Offline Reinforcement Learning Using Financial Algorithmic Approaches](https://mlsanity.com/paper/2502.10473/) (2025)
- [Learning to Focus: Prioritizing Informative Histories with Structured Attention Mechanisms in Partially Observable Reinforcement Learning](https://mlsanity.com/paper/2511.06946/) (2025)
- [Large Language Models and 3D Vision for Intelligent Robotic Perception and Autonomy](https://mlsanity.com/paper/2511.11777/) (2025)

## Abstract (arXiv)

Recent developments in Transformers have opened new interesting areas of research in partially observable reinforcement learning tasks. Results from late 2019 showed that Transformers are able to outperform LSTMs on both memory intense and reactive tasks. In this work we first partially replicate the results shown in Stabilizing Transformers in RL on both reactive and memory based environments. We then show performance improvement coupled with reduced computation when adding adaptive attention span to this Stable Transformer on a challenging DMLab30 environment. The code for all our experiments and models is available at https://github.com/jerrodparker20/adaptive-transformers-in-rl.