mlsanityData to 14 Oct 2026

Vintix II: Decision Pre-Trained Transformer is a Scalable In-Context Reinforcement Learner

arXiv 2604.05112 ↗First posted 6 Apr 2026cs.LG cs.AI cs.ROcited by 1

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

Recent progress in in-context reinforcement learning (ICRL) has demonstrated its potential for training generalist agents that can acquire new tasks directly at inference. Algorithm Distillation (AD) pioneered this paradigm and was subsequently scaled to multi-domain settings, although its ability to generalize to unseen tasks remained limited. The Decision Pre-Trained Transformer (DPT) was introduced as an alternative, showing stronger in-context reinforcement learning abilities in simplified domains, but its scalability had not been established. In this work, we extend DPT to diverse multi-domain environments, applying Flow Matching as a natural training choice that preserves its interpretation as Bayesian posterior sampling. As a result, we obtain an agent trained across hundreds of diverse tasks that achieves clear gains in generalization to the held-out test set. This agent improves upon prior AD scaling and demonstrates stronger performance in both online and offline inference, reinforcing ICRL as a viable alternative to expert distillation for training generalist agents.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 70 of its 84 references that are in the map. First posted on arXiv 6 Apr 2026.

Builds on

70 of 84 references.

Closest ancestors · 10

Same direction, newest first

  1. 2025The Impact of On-Policy Parallelized Data Collection on Deep Reinforcement Learning Networks3 Jun
  2. 2025Filtering Learning Histories Enhances In-Context Reinforcement Learning21 May
  3. 2025Zero-Shot Whole-Body Humanoid Control via Behavioral Foundation Models15 Apr
  4. 2025Distilling Reinforcement Learning Algorithms for In-Context Model-Based Planning26 Feb
  5. 2025A Survey of In-Context Reinforcement Learning11 Feb
  6. 2025Vintix: Action Model via In-Context Reinforcement Learning31 Jan
  7. 2024AMAGO-2: Breaking the Multi-Task Barrier in Meta-Reinforcement Learning with Transformers17 Nov
  8. 2024N-Gram Induction Heads for In-Context RL: Improving Stability and Reducing Data Needs4 Nov
  9. 2024XLand-100B: A Large-Scale Multi-Task Dataset for In-Context Reinforcement Learning13 Jun
  10. 2024MAMBA: an Effective World Model Approach for Meta-Reinforcement Learning14 Mar

Other references · 60

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  4. 2023GPT-4 Technical Reportcited by 18,396
  5. 2017Proximal Policy Optimization Algorithmscited by 15,215
  6. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  7. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  8. 2022Flow Matching for Generative Modelingcited by 5,702
  9. 2025Qwen2.5-VL Technical Reportcited by 5,587
  10. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  11. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
  12. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,700
38 more
  1. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  2. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
  3. 2022Training Compute-Optimal Large Language Modelscited by 3,019
  4. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
  5. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,838
  6. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  7. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,407
  8. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,234
  9. 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,761
  10. 2016OpenAI Gymcited by 1,736
  11. 2024Octo: An Open-Source Generalist Robot Policycited by 1,484
  12. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,204
  13. 2022Planning with Diffusion for Flexible Behavior Synthesiscited by 1,061
  14. 2022A Generalist Agentcited by 808
  15. 2021What Matters in Learning from Offline Human Demonstrations for Robot Manipulationcited by 725
  16. 2025FAST: Efficient Action Tokenization for Vision-Language-Action Modelscited by 665
  17. 2022Diffusion Policies as an Expressive Policy Class for Offline Reinforcement Learningcited by 562
  18. 2025SmolVLA: A Vision-Language-Action Model for Affordable and Efficient Roboticscited by 530
  19. 2016RL2: Fast Reinforcement Learning via Slow Reinforcement Learningcited by 503
  20. 2024TinyVLA: Towards Fast, Data-Efficient Vision-Language-Action Models for Robotic Manipulationcited by 339
  21. 2021MetaDrive: Composing Diverse Driving Scenarios for Generalizable Reinforcement Learningcited by 290
  22. 2022In-context Reinforcement Learning with Algorithm Distillationcited by 190
  23. 2024Mobile-Agent-v2: Mobile Device Operation Assistant with Effective Navigation via Multi-Agent Collaborationcited by 189
  24. 2025Flow Q-Learningcited by 184
  25. 2023Supervised Pretraining Can Learn In-Context Reinforcement Learningcited by 139
  26. 2022Towards Human-Level Bimanual Dexterous Manipulation with Reinforcement Learningcited by 119
  27. 2023Diffusion Model is an Effective Planner and Data Synthesizer for Multi-Task Reinforcement Learningcited by 115
  28. 2024Diffusion Actor-Critic with Entropy Regulatorcited by 94
  29. 2023Diffusion Models for Reinforcement Learning: A Surveycited by 91
  30. 2018GPU-Accelerated Robotic Simulation for Distributed Reinforcement Learningcited by 90
  31. 2025Energy-Weighted Flow Matching for Offline Reinforcement Learningcited by 73
  32. 2024Chinchilla Scaling: A replication attemptcited by 72
  33. 2023MetaDiffuser: Diffusion Model as Conditional Planner for Offline Meta-RLcited by 68
  34. 2024Gymnasium: A Standard Interface for Reinforcement Learning Environmentscited by 46
  35. 2024Kinetix: Investigating the Training of General Agents through Open-Ended Physics-Based Control Taskscited by 45
  36. 2023AMAGO: Scalable In-Context Reinforcement Learning for Adaptive Agentscited by 41
  37. 2021Procedural Generalization by Planning with Self-Supervised World Modelscited by 31
  38. 2023Emergence of In-Context Reinforcement Learning from Noise Distillationcited by 27

50 most cited of 60 All 60 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026In-Context Reinforcement Learning under Non-Stationarity: A Surveycited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2023Transformers as Decision Makers: Provable In-Context Reinforcement Learning via Supervised Pretraining
  2. 2026Transformers Provably Implement In-Context Reinforcement Learning with Policy Improvement
  3. 2024In-Context Decision Transformer: Reinforcement Learning via Hierarchical Chain-of-Thought
  4. 2025Heuristic Transformer: Belief Augmented In-Context Reinforcement Learning
  5. 2026In-Context Reinforcement Learning From Suboptimal Historical Data
  6. 2026One for All: A Non-Linear Transformer can Enable Cross-Domain Generalization for In-Context Reinforcement Learning
  7. 2024Random Policy Enables In-Context Reinforcement Learning within Trust Horizons
  8. 2025RL + Transformer = A General-Purpose Problem Solver