mlsanityData to 14 Oct 2026

Learning to reinforcement learn

arXiv 1611.05763 ↗First posted 17 Nov 2016cs.LG cs.AI stat.MLcited by 377

Not on the map. Why?

Cited by 70 in Deep reinforcement learning methods, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

In recent years deep reinforcement learning (RL) systems have attained superhuman performance in a number of challenging task domains. However, a major limitation of such applications is their demand for massive amounts of training data. A critical present objective is thus to develop deep RL methods that can adapt rapidly to new tasks. In the present work we introduce a novel approach to this challenge, which we refer to as deep meta-reinforcement learning. Previous work has shown that recurrent networks can support meta-learning in a fully supervised context. We extend this approach to the RL setting. What emerges is a system that is trained using one RL algorithm, but whose recurrent dynamics implement a second, quite separate RL procedure. This second, learned RL algorithm can differ from the original one in arbitrary ways. Importantly, because it is learned, it is configured to exploit structure in the training domain. We unpack these points in a series of seven proof-of-concept experiments, each of which examines a key aspect of deep meta-RL. We consider prospects for extending and scaling up the approach, and also point out some potentially important implications for neuroscience.

Key facts. Cited by 377 papers in the map (data through 14 Oct 2026). First posted on arXiv 17 Nov 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2018: 2; 2019: 3; 2020: 2; 2021: 83; 2022: 81; 2023: 67; 2024: 57; 2025: 48; 2026: 34.

377 papers.

Sort:

By area:

  1. 2024Automated Design of Agentic Systemscited by 407
  2. 2022Data Distributional Properties Drive Emergent In-Context Learning in Transformerscited by 309
  3. 2021MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scalecited by 229
  4. 2022In-context Reinforcement Learning with Algorithm Distillationcited by 190
  5. 2021A Survey of Zero-shot Generalisation in Deep Reinforcement Learningcited by 170
  6. 2023Foundation Models for Decision Making: Problems, Methods, and Opportunitiescited by 164
  7. 2021Investigating Bi-Level Optimization for Learning and Vision from a Unified Perspective: A Survey and Beyondcited by 153
  8. 2023Supervised Pretraining Can Learn In-Context Reinforcement Learningcited by 139
  9. 2023Human-Timescale Adaptation in an Open-Ended Task Spacecited by 138
  10. 2021Recurrent Model-Free RL Can Be a Strong Baseline for Many POMDPscited by 122
  11. 2021Advances and Challenges in Conversational Recommender Systems: A Surveycited by 118
  12. 2023Structured State Space Models for In-Context Reinforcement Learningcited by 111
38 more
  1. 2021Generalized Decision Transformer for Offline Hindsight Information Matchingcited by 106
  2. 2022General-Purpose In-Context Learning by Meta-Learning Transformerscited by 105
  3. 2023Uncovering mesa-optimization algorithms in Transformerscited by 100
  4. 2020Offline Meta-Reinforcement Learning with Advantage Weightingcited by 93
  5. 2022Evaluating Human-Language Model Interactioncited by 92
  6. 2022Few-Shot Preference Learning for Human-in-the-Loop RLcited by 87
  7. 2025Self-Adapting Language Modelscited by 84
  8. 2022A Walk in the Park: Learning to Walk in 20 Minutes With Model-Free Reinforcement Learningcited by 83
  9. 2023Transformers as Decision Makers: Provable In-Context Reinforcement Learning via Supervised Pretrainingcited by 83
  10. 2022Towards Learning Universal Hyperparameter Optimizers with Transformerscited by 81
  11. 2021Asymmetric self-play for automatic goal discovery in robotic manipulationcited by 72
  12. 2022Automated Reinforcement Learning (AutoRL): A Survey and Open Problemscited by 70
  13. 2021Robot Learning from Randomized Simulations: A Reviewcited by 63
  14. 2021Shaking the foundations: delusions in sequence models for interaction and controlcited by 62
  15. 2021A Survey of Deep RL and IL for Autonomous Driving Policy Learningcited by 61
  16. 2022Pre-Trained Image Encoder for Generalizable Visual Reinforcement Learningcited by 60
  17. 2023Meta-in-context learning in large language modelscited by 59
  18. 2022Sharp-MAML: Sharpness-Aware Model-Agnostic Meta Learningcited by 58
  19. 2023XLand-MiniGrid: Scalable Meta-Reinforcement Learning Environments in JAXcited by 55
  20. 2021Reinforcement Learning for Selective Key Applications in Power Systems: Recent Advances and Future Challengescited by 50
  21. 2021Generalization in Dexterous Manipulation via Geometry-Aware Multi-Task Learningcited by 50
  22. 2021Evolving Reinforcement Learning Algorithmscited by 49
  23. 2022Contextualize Me -- The Case for Context in Reinforcement Learningcited by 46
  24. 2021Bootstrapped Meta-Learningcited by 45
  25. 2021Bayesian decision-making under misspecified priors with applications to meta-learningcited by 44
  26. 2022Discovering Evolution Strategies via Meta-Black-Box Optimizationcited by 44
  27. 2021SECANT: Self-Expert Cloning for Zero-Shot Generalization of Visual Policiescited by 43
  28. 2022A Modern Self-Referential Weight Matrix That Learns to Modify Itselfcited by 41
  29. 2022Transformers are Meta-Reinforcement Learnerscited by 41
  30. 2023AMAGO: Scalable In-Context Reinforcement Learning for Adaptive Agentscited by 41
  31. 2025LLMs are Greedy Agents: Effects of RL Fine-tuning on Decision-Making Abilitiescited by 41
  32. 2021The Information Geometry of Unsupervised Reinforcement Learningcited by 37
  33. 2022Skill-based Meta-Reinforcement Learningcited by 37
  34. 2022Invariant and Transportable Representations for Anti-Causal Domain Shiftscited by 37
  35. 2018Learning a Prior over Intent via Meta-Inverse Reinforcement Learningcited by 36
  36. 2021Alchemy: A benchmark and analysis toolkit for meta-reinforcement learning agentscited by 36
  37. 2021Fast and Slow Learning of Recurrent Independent Mechanismscited by 36
  38. 2024The broader spectrum of in-context learningcited by 35

50 most cited of 377 All 377 (JSON)