mlsanityData to 7 Oct 2026

Vintix: Action Model via In-Context Reinforcement Learning

arXiv 2501.19400 ↗First posted 31 Jan 2025cs.LG cs.AI cs.ROcited by 8

Not on the map. Why?

Abstract (arXiv)

In-Context Reinforcement Learning (ICRL) represents a promising paradigm for developing generalist agents that learn at inference time through trial-and-error interactions, analogous to how large language models adapt contextually, but with a focus on reward maximization. However, the scalability of ICRL beyond toy tasks and single-domain settings remains an open challenge. In this work, we present the first steps toward scaling ICRL by introducing a fixed, cross-domain model capable of learning behaviors through in-context reinforcement learning. Our results demonstrate that Algorithm Distillation, a framework designed to facilitate ICRL, offers a compelling and competitive alternative to expert distillation to construct versatile action models. These findings highlight the potential of ICRL as a scalable approach for generalist decision-making systems. Code released at https://github.com/dunnolab/vintix

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 54 of its 69 references that are in the map. First posted on arXiv 31 Jan 2025.

Builds on

54 of 69 references.

References · 54

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  3. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  4. 2020Longformer: The Long-Document Transformercited by 3,362
  5. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  6. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  7. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,528
  8. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,413
  9. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,327
  10. 2020Gradient Surgery for Multi-Task Learningcited by 1,210
  11. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,168
  12. 2021Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolationcited by 1,085
38 more
  1. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,084
  2. 2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 866
  3. 2018Multi-Task Learning as Multi-Objective Optimizationcited by 809
  4. 2021Offline Reinforcement Learning as One Big Sequence Modeling Problemcited by 636
  5. 2016Learning to learn by gradient descent by gradient descentcited by 616
  6. 2016RL2: Fast Reinforcement Learning via Slow Reinforcement Learningcited by 490
  7. 2019Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variablescited by 372
  8. 2022Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videoscited by 338
  9. 2020Agent57: Outperforming the Atari Human Benchmarkcited by 300
  10. 2019Challenges of Real-World Reinforcement Learningcited by 282
  11. 2022Black-Box Tuning for Language-Model-as-a-Servicecited by 254
  12. 2018The Natural Language Decathlon: Multitask Learning as Question Answeringcited by 235
  13. 2022Multi-Game Decision Transformerscited by 212
  14. 2021Multi-Task Reinforcement Learning with Context-based Representationscited by 169
  15. 2020Gradient Vaccine: Investigating and Improving Multi-task Optimization in Massively Multilingual Modelscited by 167
  16. 2024Scaling Proprioceptive-Visual Learning with Heterogeneous Pre-trained Transformerscited by 145
  17. 2018Multi-task Deep Reinforcement Learning with PopArtcited by 138
  18. 2023Supervised Pretraining Can Learn In-Context Reinforcement Learningcited by 137
  19. 2023Human-Timescale Adaptation in an Open-Ended Task Spacecited by 136
  20. 2019Reinforcement Learning Upside Down: Don't Predict Rewards -- Just Map Them to Actionscited by 124
  21. 2022CORL: Research-oriented Deep Offline Reinforcement Learning Librarycited by 117
  22. 2022Towards Human-Level Bimanual Dexterous Manipulation with Reinforcement Learningcited by 113
  23. 2024BAKU: An Efficient Transformer for Multi-Task Policy Learningcited by 80
  24. 2018GPU-Accelerated Robotic Simulation for Distributed Reinforcement Learningcited by 77
  25. 2020Tasks, stability, architecture, and compute: Training more effective learned optimizers, and using them to train themselvescited by 57
  26. 2022VeLO: Training Versatile Learned Optimizers by Scaling Upcited by 57
  27. 2017A Benchmark Environment Motivated by Industrial Control Problemscited by 31
  28. 2021Procedural Generalization by Planning with Self-Supervised World Modelscited by 30
  29. 2023Emergence of In-Context Reinforcement Learning from Noise Distillationcited by 26
  30. 2024Reinformer: Max-Return Sequence Modeling for Offline RLcited by 25
  31. 2021A Generalizable Approach to Learning Optimizerscited by 23
  32. 2022Causes and Cures for Interference in Multilingual Translationcited by 23
  33. 2020MELD: Meta-Reinforcement Learning from Images via Latent State Modelscited by 22
  34. 2021Robust Optimization for Multilingual Translation with Imbalanced Datacited by 15
  35. 2024Can Learned Optimization Make Reinforcement Learning Less Difficult?cited by 14
  36. 2024A Large Recurrent Action Model: xLSTM enables Fast Inference for Robotics Taskscited by 14
  37. 2024AMAGO-2: Breaking the Multi-Task Barrier in Meta-Reinforcement Learning with Transformerscited by 14
  38. 2024XLand-100B: A Large-Scale Multi-Task Dataset for In-Context Reinforcement Learningcited by 13

50 most cited of 54 All 54 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 5; 2026: 3.

8 papers.

  1. 2025Vision-Language-Action (VLA) Models: Concepts, Progress, Applications and Challengescited by 72
  2. 2025Yes, Q-learning Helps Offline In-Context RLcited by 8
  3. 2025Zero-Shot Adaptation of Behavioral Foundation Models to Unseen Dynamicscited by 8
  4. 2025Towards Provable Emergence of In-Context Reinforcement Learningcited by 2
  5. 2026Benchmarking the Limits of In-Context Reinforcement Learning for Ad-Hoc Teamworkcited by 2
  6. 2026IntentScore: Intent-Conditioned Action Evaluation for Computer-Use Agentscited by 1
  7. 2025Robust In-Context Reinforcement Learning Under Reward Poisoning Attackscited by 0
  8. 2026Latent Q-Barrier Shielding for Safe In-Context Reinforcement Learningcited by 0