mlsanityData to 7 Oct 2026

Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variables

arXiv 1903.08254 ↗First posted 19 Mar 2019cs.LG cs.AI stat.MLcited by 372

Not on the map. Why?

Cited by 47 in Offline RL and imitation learning from data, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Deep reinforcement learning algorithms require large amounts of experience to learn an individual task. While in principle meta-reinforcement learning (meta-RL) algorithms enable agents to learn new skills from small amounts of experience, several major challenges preclude their practicality. Current methods rely heavily on on-policy experience, limiting their sample efficiency. The also lack mechanisms to reason about task uncertainty when adapting to new tasks, limiting their effectiveness in sparse reward problems. In this paper, we address these challenges by developing an off-policy meta-RL algorithm that disentangles task inference and control. In our approach, we perform online probabilistic filtering of latent task variables to infer how to solve a new task from small amounts of experience. This probabilistic interpretation enables posterior sampling for structured and efficient exploration. We demonstrate how to integrate these task variables with off-policy RL algorithms to achieve both meta-training and adaptation efficiency. Our method outperforms prior algorithms in sample efficiency by 20-100X as well as in asymptotic performance on several meta-RL benchmarks.

Key facts. Cited by 372 papers in the map (data through 7 Oct 2026). First posted on arXiv 19 Mar 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2019: 3; 2020: 3; 2021: 74; 2022: 77; 2023: 60; 2024: 54; 2025: 50; 2026: 51.

372 papers.

Sort:

By area:

  1. 2021MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scalecited by 226
  2. 2021How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learnedcited by 201
  3. 2022In-context Reinforcement Learning with Algorithm Distillationcited by 186
  4. 2023Supervised Pretraining Can Learn In-Context Reinforcement Learningcited by 137
  5. 2023Human-Timescale Adaptation in an Open-Ended Task Spacecited by 136
  6. 2023Large Language Models for Robotics: A Surveycited by 132
  7. 2021Recurrent Model-Free RL Can Be a Strong Baseline for Many POMDPscited by 115
  8. 2025Towards System 2 Reasoning in LLMs: Learning How to Think With Meta Chain-of-Thoughtcited by 108
  9. 2021Generalized Decision Transformer for Offline Hindsight Information Matchingcited by 105
  10. 2020Offline Meta-Reinforcement Learning with Advantage Weightingcited by 92
  11. 2022Few-Shot Preference Learning for Human-in-the-Loop RLcited by 85
  12. 2023Transformers as Decision Makers: Provable In-Context Reinforcement Learning via Supervised Pretrainingcited by 82
38 more
  1. 2022A Walk in the Park: Learning to Walk in 20 Minutes With Model-Free Reinforcement Learningcited by 76
  2. 2020Fighting Copycat Agents in Behavioral Cloning from Observation Historiescited by 70
  3. 2022CACTI: A Framework for Scalable Multi-Task Multi-Scene Visual Imitation Learningcited by 70
  4. 2022Automated Reinforcement Learning (AutoRL): A Survey and Open Problemscited by 69
  5. 2021A Survey of Exploration Methods in Reinforcement Learningcited by 68
  6. 2023MetaDiffuser: Diffusion Model as Conditional Planner for Offline Meta-RLcited by 65
  7. 2022Challenges and Opportunities in Offline Reinforcement Learning from Visual Observationscited by 63
  8. 2024Efficient Online Reinforcement Learning Fine-Tuning Need Not Retain Offline Datacited by 58
  9. 2021AdaAfford: Learning to Adapt Manipulation Affordance for 3D Articulated Objects via Few-shot Interactionscited by 56
  10. 2021Offline Meta-Reinforcement Learning with Online Self-Supervisioncited by 52
  11. 2019Meta-Inverse Reinforcement Learning with Probabilistic Context Variablescited by 50
  12. 2021Generalization in Dexterous Manipulation via Geometry-Aware Multi-Task Learningcited by 49
  13. 2021AdaRL: What, Where, and How to Adapt in Transfer Reinforcement Learningcited by 46
  14. 2021Augmented World Models Facilitate Zero-Shot Dynamics Generalization From a Single Offline Environmentcited by 41
  15. 2022Contextualize Me -- The Case for Context in Reinforcement Learningcited by 41
  16. 2021Baby Intuitions Benchmark (BIB): Discerning the goals, preferences, and actions of otherscited by 39
  17. 2022Transformers are Meta-Reinforcement Learnerscited by 39
  18. 2022Hypernetworks in Meta-Reinforcement Learningcited by 39
  19. 2021Interesting Object, Curious Agent: Learning Task-Agnostic Explorationcited by 38
  20. 2021Offline Meta-Reinforcement Learning for Industrial Insertioncited by 37
  21. 2021Tell me why! Explanations support learning relational and causal structurecited by 36
  22. 2022Skill-based Meta-Reinforcement Learningcited by 36
  23. 2023AMAGO: Scalable In-Context Reinforcement Learning for Adaptive Agentscited by 36
  24. 2021Alchemy: A benchmark and analysis toolkit for meta-reinforcement learning agentscited by 35
  25. 2026MetaClaw: Just Talk -- An Agent That Meta-Learns and Evolves in the Wildcited by 35
  26. 2021Deep reinforcement learning in medical imaging: A literature reviewcited by 34
  27. 2021Recomposing the Reinforcement Learning Building Blocks with Hypernetworkscited by 33
  28. 2022Robust Task Representations for Offline Meta-Reinforcement Learning via Contrastive Learningcited by 33
  29. 2023End-to-End Meta-Bayesian Optimisation with Transformer Neural Processescited by 31
  30. 2024Unsupervised Zero-Shot Reinforcement Learning via Functional Reward Encodingscited by 31
  31. 2021Procedural Generalization by Planning with Self-Supervised World Modelscited by 30
  32. 2023Inverse Dynamics Pretraining Learns Good Representations for Multitask Imitationcited by 30
  33. 2023Cross-Domain Policy Adaptation via Value-Guided Data Filteringcited by 30
  34. 2021Coach-Player Multi-Agent Reinforcement Learning for Dynamic Team Compositioncited by 28
  35. 2022Don't Start From Scratch: Leveraging Prior Data to Automate Robotic Reinforcement Learningcited by 28
  36. 2019Watch, Try, Learn: Meta-Learning from Demonstrations and Rewardcited by 26
  37. 2022Context is Everything: Implicit Identification for Dynamics Adaptationcited by 26
  38. 2024FuRL: Visual-Language Models as Fuzzy Rewards for Reinforcement Learningcited by 26

50 most cited of 372 All 372 (JSON)