mlsanityData to 14 Oct 2026

RL + Transformer = A General-Purpose Problem Solver

arXiv 2501.14176 ↗First posted 24 Jan 2025cs.LG cs.AIcited by 3

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

What if artificial intelligence could not only solve problems for which it was trained but also learn to teach itself to solve new problems (i.e., meta-learn)? In this study, we demonstrate that a pre-trained transformer fine-tuned with reinforcement learning over multiple episodes develops the ability to solve problems that it has never encountered before - an emergent ability called In-Context Reinforcement Learning (ICRL). This powerful meta-learner not only excels in solving unseen in-distribution environments with remarkable sample efficiency, but also shows strong performance in out-of-distribution environments. In addition, we show that it exhibits robustness to the quality of its training data, seamlessly stitches together behaviors from its context, and adapts to non-stationary environments. These behaviors demonstrate that an RL-trained transformer can iteratively improve upon its own solutions, making it an excellent general-purpose problem solver.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 9 of its 16 references that are in the map. First posted on arXiv 24 Jan 2025.

Builds on

9 of 16 references.

References · 9

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  3. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,920
  4. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,745
  5. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,546
  6. 2022Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learningcited by 896
  7. 2016RL2: Fast Reinforcement Learning via Slow Reinforcement Learningcited by 503
  8. 2023Supervised Pretraining Can Learn In-Context Reinforcement Learningcited by 139
  9. 2024Robust Offline Imitation Learning from Diverse Auxiliary Datacited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 1.

3 papers.

  1. 2025Exploitation Is All You Need... for Explorationcited by 0
  2. 2025Accelerating Transformers in Online RLcited by 0
  3. 2026Meta-Learning and Meta-Reinforcement Learning -- Tracing the Path towards DeepMind's Adaptive Agentcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Transformers Provably Implement In-Context Reinforcement Learning with Policy Improvement
  2. 2024Transformers Can Learn Temporal Difference Methods for In-Context Reinforcement Learning
  3. 2023Transformers in Reinforcement Learning: A Survey
  4. 2023Transformers as Decision Makers: Provable In-Context Reinforcement Learning via Supervised Pretraining
  5. 2026Vintix II: Decision Pre-Trained Transformer is a Scalable In-Context Reinforcement Learner
  6. 2026One for All: A Non-Linear Transformer can Enable Cross-Domain Generalization for In-Context Reinforcement Learning
  7. 2025Heuristic Transformer: Belief Augmented In-Context Reinforcement Learning
  8. 2024AMAGO-2: Breaking the Multi-Task Barrier in Meta-Reinforcement Learning with Transformers