mlsanityData to 7 Oct 2026

Improved Robustness and Safety for Pre-Adaptation of Meta Reinforcement Learning with Prior Regularization

arXiv 2108.08448 ↗First posted 19 Aug 2021cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

Meta Reinforcement Learning (Meta-RL) has seen substantial advancements recently. In particular, off-policy methods were developed to improve the data efficiency of Meta-RL techniques. textitProbabilistic embeddings for actor-critic RL (PEARL) is a leading approach for multi-MDP adaptation problems. A major drawback of many existing Meta-RL methods, including PEARL, is that they do not explicitly consider the safety of the prior policy when it is exposed to a new task for the first time. Safety is essential for many real-world applications, including field robots and Autonomous Vehicles (AVs). In this paper, we develop the PEARL PLUS (PEARL^+) algorithm, which optimizes the policy for both prior (pre-adaptation) safety and posterior (after-adaptation) performance. Building on top of PEARL, our proposed PEARL^+ algorithm introduces a prior regularization term in the reward function and a new Q-network for recovering the state-action value under prior context assumptions, to improve the robustness to task distribution shift and safety of the trained network exposed to a new task for the first time. The performance of PEARL^+ is validated by solving three safety-critical problems related to robots and AVs, including two MuJoCo benchmark problems. From the simulation experiments, we show that safety of the prior policy is significantly improved and more robust to task distribution shift compared to PEARL.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 21 of its 27 references that are in the map. First posted on arXiv 19 Aug 2021.

Builds on

21 of 27 references.

References · 21

  1. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  3. 2018On First-Order Meta-Learning Algorithmscited by 1,003
  4. 2016Deep Variational Information Bottleneckcited by 915
  5. 2016RL2: Fast Reinforcement Learning via Slow Reinforcement Learningcited by 490
  6. 2019Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variablescited by 372
  7. 2016Learning to reinforcement learncited by 369
  8. 2018How to train your MAMLcited by 306
  9. 2019Adversarial Attacks on Graph Neural Networks via Meta Learningcited by 261
  10. 2018Meta-Reinforcement Learning of Structured Exploration Strategiescited by 128
  11. 2019Improving Generalization in Meta Reinforcement Learning using Learned Objectivescited by 68
  12. 2018Weakly-Convex Concave Min-Max Optimization: Provable Algorithms and Applications in Machine Learningcited by 62
9 more
  1. 2019Meta-Q-Learningcited by 62
  2. 2020Safe Reinforcement Learning for Autonomous Vehicles through Parallel Constrained Policy Optimizationcited by 34
  3. 2020Model-based Adversarial Meta-Reinforcement Learningcited by 32
  4. 2020Model-Based Meta-Reinforcement Learning for Flight with Suspended Payloadscited by 27
  5. 2019Guided Meta-Policy Searchcited by 26
  6. 2020Online Meta-Critic Learning for Off-Policy Actor-Critic Methodscited by 17
  7. 2021Meta-Adversarial Inverse Reinforcement Learning for Decision-making Taskscited by 6
  8. 2017Composing Meta-Policies for Autonomous Driving Using Hierarchical Deep Reinforcement Learningcited by 4
  9. 2021Quick Learner Automated Vehicle Adapting its Roadmanship to Varying Traffic Cultures with Meta Reinforcement Learningcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 3.

4 papers.

  1. 2026Safe Continual Reinforcement Learning Methods for Nonstationary Environments. Towards a Survey of the State of the Artcited by 1
  2. 2025TLXML: Task-Level Explanation of Meta-Learning via Influence Functionscited by 0
  3. 2026Adjustment Speed as a Safety Constraint for Nonstationary Reinforcement Learningcited by 0
  4. 2026Proactive Context-Forecasted Safety Constraints for Nonstationary Reinforcement Learningcited by 0