mlsanityData to 7 Oct 2026

VariBAD: A Very Good Method for Bayes-Adaptive Deep RL via Meta-Learning

arXiv 1910.08348 ↗First posted 18 Oct 2019cs.LG stat.MLcited by 210

Not on the map. Why?

Cited by 26 in Offline RL and imitation learning from data, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Trading off exploration and exploitation in an unknown environment is key to maximising expected return during learning. A Bayes-optimal policy, which does so optimally, conditions its actions not only on the environment state but on the agent's uncertainty about the environment. Computing a Bayes-optimal policy is however intractable for all but the smallest tasks. In this paper, we introduce variational Bayes-Adaptive Deep RL (variBAD), a way to meta-learn to perform approximate inference in an unknown environment, and incorporate task uncertainty directly during action selection. In a grid-world domain, we illustrate how variBAD performs structured online exploration as a function of task uncertainty. We further evaluate variBAD on MuJoCo domains widely used in meta-RL and show that it achieves higher online return than existing methods.

Key facts. Cited by 210 papers in the map (data through 7 Oct 2026). First posted on arXiv 18 Oct 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 1; 2021: 34; 2022: 43; 2023: 47; 2024: 41; 2025: 26; 2026: 18.

210 papers.

Sort:

By area:

  1. 2021MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scalecited by 226
  2. 2021A Survey of Zero-shot Generalisation in Deep Reinforcement Learningcited by 164
  3. 2022A Comprehensive Survey of Few-shot Learning: Evolution, Applications, Challenges, and Opportunitiescited by 162
  4. 2023Supervised Pretraining Can Learn In-Context Reinforcement Learningcited by 137
  5. 2023Human-Timescale Adaptation in an Open-Ended Task Spacecited by 136
  6. 2021Recurrent Model-Free RL Can Be a Strong Baseline for Many POMDPscited by 115
  7. 2023Diffusion Model is an Effective Planner and Data Synthesizer for Multi-Task Reinforcement Learningcited by 114
  8. 2021Why Generalization in RL is Difficult: Epistemic POMDPs and Implicit Partial Observabilitycited by 102
  9. 2020Offline Meta-Reinforcement Learning with Advantage Weightingcited by 92
  10. 2023Transformers as Decision Makers: Provable In-Context Reinforcement Learning via Supervised Pretrainingcited by 82
  11. 2022Automated Reinforcement Learning (AutoRL): A Survey and Open Problemscited by 69
  12. 2023XLand-MiniGrid: Scalable Meta-Reinforcement Learning Environments in JAXcited by 54
38 more
  1. 2021Offline Meta-Reinforcement Learning with Online Self-Supervisioncited by 52
  2. 2022Offline RL Policies Should be Trained to be Adaptivecited by 52
  3. 2021Augmented World Models Facilitate Zero-Shot Dynamics Generalization From a Single Offline Environmentcited by 41
  4. 2022Transformers are Meta-Reinforcement Learnerscited by 39
  5. 2022Skill-based Meta-Reinforcement Learningcited by 36
  6. 2021Alchemy: A benchmark and analysis toolkit for meta-reinforcement learning agentscited by 35
  7. 2022Robust Task Representations for Offline Meta-Reinforcement Learning via Contrastive Learningcited by 33
  8. 2022Choreographer: Learning and Adapting Skills in Imaginationcited by 31
  9. 2023Cross-Domain Policy Adaptation via Value-Guided Data Filteringcited by 30
  10. 2021Deep Interactive Bayesian Reinforcement Learning via Meta-Learningcited by 29
  11. 2022Don't Start From Scratch: Leveraging Prior Data to Automate Robotic Reinforcement Learningcited by 28
  12. 2022You Only Live Once: Single-Life Reinforcement Learningcited by 23
  13. 2023State Regularized Policy Optimization on Data with Dynamics Shiftcited by 23
  14. 2023Maximum State Entropy Exploration using Predecessor and Successor Representationscited by 23
  15. 2022Learning Options via Compressioncited by 22
  16. 2023Accelerating Exploration with Unlabeled Prior Datacited by 21
  17. 2024Deep Generative Models for Offline Policy Learning: Tutorial, Survey, and Perspectives on Future Directionscited by 20
  18. 2022Deciding What to Model: Value-Equivalent Sampling for Reinforcement Learningcited by 19
  19. 2023Prompt-Tuning Decision Transformer with Preference Rankingcited by 19
  20. 2022Influencing Long-Term Behavior in Multiagent Reinforcement Learningcited by 18
  21. 2023Dynamics Generalisation in Reinforcement Learning via Adaptive Context-Aware Policiescited by 18
  22. 2021Unsupervised Reinforcement Learning in Multiple Environmentscited by 17
  23. 2023RePo: Resilient Model-Based Reinforcement Learning by Regularizing Posterior Predictabilitycited by 17
  24. 2023Context Shift Reduction for Offline Meta-Reinforcement Learningcited by 17
  25. 2024Equivariant Reinforcement Learning under Partial Observabilitycited by 17
  26. 2021Bayesian Bellman Operatorscited by 16
  27. 2022Robust Policy Learning over Multiple Uncertainty Setscited by 16
  28. 2022Grounding Aleatoric Uncertainty for Unsupervised Environment Designcited by 16
  29. 2023Discovering General Reinforcement Learning Algorithms with Adversarial Environment Designcited by 16
  30. 2024Towards an Information Theoretic Framework of Context-Based Offline Meta-Reinforcement Learningcited by 16
  31. 2024A Comprehensive Survey of Cross-Domain Policy Transfer for Embodied Agentscited by 16
  32. 2021CoMPS: Continual Meta Policy Searchcited by 15
  33. 2022An Adaptive Deep RL Method for Non-Stationary Environments with Piecewise Stable Contextcited by 15
  34. 2023Recurrent Hypernetworks are Surprisingly Strong in Meta-RLcited by 15
  35. 2021Online Limited Memory Neural-Linear Bandits with Likelihood Matchingcited by 14
  36. 2022Model-Based Offline Meta-Reinforcement Learning with Regularizationcited by 14
  37. 2023Memory-Based Meta-Learning on Non-Stationary Distributionscited by 14
  38. 2021Continual Learning In Environments With Polynomial Mixing Timescited by 13

50 most cited of 210 All 210 (JSON)