mlsanityData to 14 Oct 2026

XLand-100B: A Large-Scale Multi-Task Dataset for In-Context Reinforcement Learning

arXiv 2406.08973 ↗First posted 13 Jun 2024cs.LG cs.AIcited by 14

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

Following the success of the in-context learning paradigm in large-scale language and computer vision models, the recently emerging field of in-context reinforcement learning is experiencing a rapid growth. However, its development has been held back by the lack of challenging benchmarks, as all the experiments have been carried out in simple environments and on small-scale datasets. We present XLand-100B, a large-scale dataset for in-context reinforcement learning based on the XLand-MiniGrid environment, as a first step to alleviate this problem. It contains complete learning histories for nearly 30,000 different tasks, covering 100B transitions and 2.5B episodes. It took 50,000 GPU hours to collect the dataset, which is beyond the reach of most academic labs. Along with the dataset, we provide the utilities to reproduce or expand it even further. We also benchmark common in-context RL baselines and show that they struggle to generalize to novel and diverse tasks. With this substantial effort, we aim to democratize research in the rapidly growing field of in-context reinforcement learning and provide a solid foundation for further scaling.

Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 54 of its 69 references that are in the map. First posted on arXiv 13 Jun 2024.

Builds on

54 of 69 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Transformers Can Learn Temporal Difference Methods for In-Context Reinforcement Learning22 May
  2. 2024Hierarchical Transformers are Efficient Meta-Reinforcement Learners9 Feb

Other references · 52

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  4. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,598
  5. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,546
  6. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,427
  7. 2014On the Properties of Neural Machine Translation: Encoder-Decoder Approachescited by 1,304
  8. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,204
  9. 2021Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolationcited by 1,103
  10. 2024Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Predictioncited by 1,023
  11. 2023Open X-Embodiment: Robotic Learning Datasets and RT-X Modelscited by 862
  12. 2022Transformers learn in-context by gradient descentcited by 732
38 more
  1. 2021Offline Reinforcement Learning as One Big Sequence Modeling Problemcited by 644
  2. 2016RL2: Fast Reinforcement Learning via Slow Reinforcement Learningcited by 503
  3. 2019Leveraging Procedural Generation to Benchmark Reinforcement Learningcited by 483
  4. 2016Learning to reinforcement learncited by 377
  5. 2021Transformers Can Do Bayesian Inferencecited by 360
  6. 2023Minigrid & Miniworld: Modular & Customizable Reinforcement Learning Environments for Goal-Oriented Taskscited by 327
  7. 2022Data Distributional Properties Drive Emergent In-Context Learning in Transformerscited by 309
  8. 2019VariBAD: A Very Good Method for Bayes-Adaptive Deep RL via Meta-Learningcited by 217
  9. 2022Multi-Game Decision Transformerscited by 214
  10. 2023Sequential Modeling Enables Scalable Learning for Large Vision Modelscited by 212
  11. 2023Large Language Models as General Pattern Machinescited by 197
  12. 2022In-context Reinforcement Learning with Algorithm Distillationcited by 190
  13. 2021Open-Ended Learning Leads to Generally Capable Agentscited by 182
  14. 2023Supervised Pretraining Can Learn In-Context Reinforcement Learningcited by 139
  15. 2023Human-Timescale Adaptation in an Open-Ended Task Spacecited by 138
  16. 2023Structured State Space Models for In-Context Reinforcement Learningcited by 111
  17. 2022General-Purpose In-Context Learning by Meta-Learning Transformerscited by 105
  18. 2024Can Mamba Learn How to Learn? A Comparative Study on In-Context Learning Taskscited by 105
  19. 2024In-Context Language Learning: Architectures and Algorithmscited by 89
  20. 2023Transformers as Decision Makers: Provable In-Context Reinforcement Learning via Supervised Pretrainingcited by 83
  21. 2023Understanding In-Context Learning in Transformers and LLMs by Learning to Learn Discrete Functionscited by 63
  22. 2023XLand-MiniGrid: Scalable Meta-Reinforcement Learning Environments in JAXcited by 55
  23. 2023A Survey on Transformers in Reinforcement Learningcited by 50
  24. 2024Is Mamba Capable of In-Context Learning?cited by 45
  25. 2023Pre-Training to Learn in Contextcited by 42
  26. 2022Transformers are Meta-Reinforcement Learnerscited by 41
  27. 2023AMAGO: Scalable In-Context Reinforcement Learning for Adaptive Agentscited by 41
  28. 2023Generalization to New Sequential Decision Making Tasks with In-Context Learningcited by 39
  29. 2023Emergent Agentic Transformer from Chain of Hindsight Experiencecited by 33
  30. 2024Linear Transformers are Versatile In-Context Learnerscited by 28
  31. 2023Emergence of In-Context Reinforcement Learning from Noise Distillationcited by 27
  32. 2024Towards Multimodal In-Context Learning for Vision & Language Modelscited by 25
  33. 2023In-Context Reinforcement Learning for Variable Action Spacescited by 24
  34. 2023Context Diffusion: In-Context Aware Image Generationcited by 21
  35. 2024Jack of All Trades, Master of Some, a Multi-Purpose Transformer Agentcited by 21
  36. 2023Recurrent Hypernetworks are Surprisingly Strong in Meta-RLcited by 16
  37. 2023Transformers in Reinforcement Learning: A Surveycited by 12
  38. 2023Cross-Episodic Curriculum for Transformer Agentscited by 10

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 4; 2025: 7; 2026: 3.

14 papers.

Sort:

By area:

  1. 2024Kinetix: Investigating the Training of General Agents through Open-Ended Physics-Based Control Taskscited by 45
  2. 2024Retrieval-Augmented Decision Transformer: External Memory for In-context RLcited by 21
  3. 2024A Large Recurrent Action Model: xLSTM enables Fast Inference for Robotics Taskscited by 14
  4. 2025Vintix: Action Model via In-Context Reinforcement Learningcited by 10
  5. 2025Yes, Q-learning Helps Offline In-Context RLcited by 10
  6. 2025Zero-Shot Adaptation of Behavioral Foundation Models to Unseen Dynamicscited by 8
  7. 2025Mixture-of-Experts Meets In-Context Reinforcement Learningcited by 8
  8. 2024N-Gram Induction Heads for In-Context RL: Improving Stability and Reducing Data Needscited by 7
  9. 2025Scalable In-Context Q-Learningcited by 5
  10. 2026Preventing Learning Stagnation in PPO by Scaling to 1 Million Parallel Environmentscited by 3
  11. 2025Multi-Agent Craftax: Benchmarking Open-Ended Multi-Agent Reinforcement Learning at the Hyperscalecited by 2
  12. 2026Benchmarking the Limits of In-Context Reinforcement Learning for Ad-Hoc Teamworkcited by 2
2 more
  1. 2025A Data-Driven Discretized CS:GO Simulation Environment to Facilitate Strategic Multi-Agent Planning Researchcited by 1
  2. 2026Vintix II: Decision Pre-Trained Transformer is a Scalable In-Context Reinforcement Learnercited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2025Reinforcement Learning for Large Model: A Survey
  2. 2026In-Context Reinforcement Learning under Non-Stationarity: A Survey
  3. 2025Towards Large-Scale In-Context Reinforcement Learning by Meta-Training in Randomized Worlds
  4. 2026Beyond Reward Engineering: A Data Recipe for Long-Context Reinforcement Learning
  5. 2023Pre-training Contextualized World Models with In-the-wild Videos for Reinforcement Learning
  6. 2024Zero-shot Model-based Reinforcement Learning using Large Language Models
  7. 2024In-context Exploration-Exploitation for Reinforcement Learning
  8. 2026In-Context Reinforcement Learning From Suboptimal Historical Data