mlsanityData to 7 Oct 2026

Distributionally Adaptive Meta Reinforcement Learning

arXiv 2210.03104 ↗First posted 6 Oct 2022cs.LG cs.AIcited by 11

Not on the map. Why?

Abstract (arXiv)

Meta-reinforcement learning algorithms provide a data-driven way to acquire policies that quickly adapt to many tasks with varying rewards or dynamics functions. However, learned meta-policies are often effective only on the exact task distribution on which they were trained and struggle in the presence of distribution shift of test-time rewards or transition dynamics. In this work, we develop a framework for meta-RL algorithms that are able to behave appropriately under test-time distribution shifts in the space of tasks. Our framework centers on an adaptive approach to distributional robustness that trains a population of meta-policies to be robust to varying levels of distribution shift. When evaluated on a potentially shifted test-time distribution of tasks, this allows us to choose the meta-policy with the most appropriate level of robustness, and use it to perform fast adaptation. We formally show how our framework allows for improved regret under distribution shift, and empirically show its efficacy on simulated robotics problems under a wide range of distribution shifts.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 40 of its 58 references that are in the map. First posted on arXiv 6 Oct 2022.

Builds on

40 of 58 references.

References · 40

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  3. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  4. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  5. 2013Auto-Encoding Variational Bayescited by 3,980
  6. 2019Certified Adversarial Robustness via Randomized Smoothingcited by 1,324
  7. 2020AWAC: Accelerating Online Reinforcement Learning with Offline Datasetscited by 647
  8. 2016RL2: Fast Reinforcement Learning via Slow Reinforcement Learningcited by 490
  9. 2017Robust Adversarial Reinforcement Learningcited by 439
  10. 2021RMA: Rapid Motor Adaptation for Legged Robotscited by 430
  11. 2019Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variablescited by 372
  12. 2017Certifying Some Distributional Robustness with Principled Adversarial Trainingcited by 333
28 more
  1. 2022Learning robust perceptive locomotion for quadrupedal robots in the wildcited by 299
  2. 2018Learning to Adapt in Dynamic, Real-World Environments Through Meta-Reinforcement Learningcited by 270
  3. 2019VariBAD: A Very Good Method for Bayes-Adaptive Deep RL via Meta-Learningcited by 210
  4. 2019Efficient Exploration via State Marginal Matchingcited by 186
  5. 2021A System for General In-Hand Object Re-Orientationcited by 155
  6. 2020Can Autonomous Vehicles Identify, Recover From, and Adapt to Distribution Shifts?cited by 139
  7. 2021Robust Reinforcement Learning on State Observations with Learned Optimal Adversarycited by 133
  8. 2022Rapid Locomotion via Reinforcement Learningcited by 130
  9. 2018Meta-Reinforcement Learning of Structured Exploration Strategiescited by 128
  10. 2020Offline Meta-Reinforcement Learning with Advantage Weightingcited by 92
  11. 2018ProMP: Proximal Meta-Policy Searchcited by 91
  12. 2020Robust Deep Reinforcement Learning through Adversarial Losscited by 73
  13. 2020Robust Reinforcement Learning using Adversarial Populationscited by 64
  14. 2018Unsupervised Meta-Learning for Reinforcement Learningcited by 38
  15. 2020Grasping with Chopsticks: Combating Covariate Shift in Model-free Imitation Learning for Fine Manipulationcited by 37
  16. 2021Generalization of Model-Agnostic Meta-Learning Algorithms: Recurring and Unseen Taskscited by 36
  17. 2019Unsupervised Curricula for Visual Meta-Reinforcement Learningcited by 33
  18. 2020Model-based Adversarial Meta-Reinforcement Learningcited by 32
  19. 2020Towards Effective Context for Meta-Reinforcement Learning: an Approach based on Contrastive Learningcited by 29
  20. 2020Exploration in Approximate Hyper-State Space for Meta Reinforcement Learningcited by 28
  21. 2020Meta-Reinforcement Learning Robust to Distributional Shift via Model Identification and Experience Relabelingcited by 24
  22. 2020MetaCURE: Meta Reinforcement Learning with Empowerment-Driven Explorationcited by 24
  23. 2020MELD: Meta-Reinforcement Learning from Images via Latent State Modelscited by 22
  24. 2022Robust Policy Learning over Multiple Uncertainty Setscited by 16
  25. 2021ProtoTransformer: A Meta-Learning Approach to Providing Student Feedbackcited by 8
  26. 2018The effects of negative adaptation in Model-Agnostic Meta-Learningcited by 6
  27. 2020Task-Robust Model-Agnostic Meta-Learningcited by 3
  28. 2020Long-Horizon Prediction and Uncertainty Propagation with Residual Point Contact Learnerscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 4; 2024: 2; 2025: 3; 2026: 2.

11 papers.

  1. 2023Provable Guarantees for Generative Behavior Cloning: Bridging Low-Level Stability and High-Level Behaviorcited by 33
  2. 2023Accelerating Exploration with Unlabeled Prior Datacited by 21
  3. 2025Model Predictive Task Sampling for Efficient and Robust Adaptationcited by 20
  4. 2023Train Hard, Fight Easy: Robust Meta Reinforcement Learningcited by 11
  5. 2025Task-Aware Virtual Training: Enhancing Generalization in Meta-Reinforcement Learning for Out-of-Distribution Taskscited by 3
  6. 2024GRAM: Generalization in Deep RL with a Robust Adaptation Modulecited by 2
  7. 2023Statistical Learning under Heterogeneous Distribution Shiftcited by 1
  8. 2024Certifiably Robust Model Evaluation in Federated Learning under Meta-Distributional Shiftscited by 0
  9. 2025An Information-Theoretic Analysis of OOD Generalization in Meta-Reinforcement Learningcited by 0
  10. 2026A Unified Causal-Origin Taxonomy of Distributional Shifts in Reinforcement Learningcited by 0
  11. 2026Inference-Time Policy Alignment for Fair Reinforcement Learningcited by 0