mlsanityData to 7 Oct 2026

MESA: Offline Meta-RL for Safe Adaptation and Fault Tolerance

arXiv 2112.03575 ↗First posted 7 Dec 2021cs.LG cs.AI cs.ROcited by 7

Not on the map. Why?

Abstract (arXiv)

Safe exploration is critical for using reinforcement learning (RL) in risk-sensitive environments. Recent work learns risk measures which measure the probability of violating constraints, which can then be used to enable safety. However, learning such risk measures requires significant interaction with the environment, resulting in excessive constraint violations during learning. Furthermore, these measures are not easily transferable to new environments. We cast safe exploration as an offline meta-RL problem, where the objective is to leverage examples of safe and unsafe behavior across a range of environments to quickly adapt learned risk measures to a new environment with previously unseen dynamics. We then propose MEta-learning for Safe Adaptation (MESA), an approach for meta-learning a risk measure for safe RL. Simulation experiments across 5 continuous control domains suggest that MESA can leverage offline data from a range of different environments to reduce constraint violations in unseen environments by up to a factor of 2 while maintaining task performance. See https://tinyurl.com/safe-meta-rl for code and supplementary material.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 35 of its 58 references that are in the map. First posted on arXiv 7 Dec 2021.

Builds on

35 of 58 references.

References · 35

  1. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  3. 2015Continuous control with deep reinforcement learningcited by 3,279
  4. 2017Constrained Policy Optimizationcited by 928
  5. 2016RL2: Fast Reinforcement Learning via Slow Reinforcement Learningcited by 490
  6. 2017Safe Reinforcement Learning via Shieldingcited by 427
  7. 2018Reward Constrained Policy Optimizationcited by 397
  8. 2019Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variablescited by 372
  9. 2016Learning to reinforcement learncited by 369
  10. 2017Safe Model-based Reinforcement Learning with Stability Guaranteescited by 347
  11. 2015Risk-Constrained Reinforcement Learning with Percentile Risk Criteriacited by 320
  12. 2018A Lyapunov-based Approach to Safe Reinforcement Learningcited by 278
23 more
  1. 2018Learning to Adapt in Dynamic, Real-World Environments Through Meta-Reinforcement Learningcited by 270
  2. 2021How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learnedcited by 201
  3. 2017Hamilton-Jacobi Reachability: A Brief Overview and Recent Advancescited by 186
  4. 2020Recovery RL: Safe Reinforcement Learning with Learned Recovery Zonescited by 156
  5. 2017A General Safety Framework for Learning-Based Control in Uncertain Robotic Systemscited by 150
  6. 2019Lyapunov-based Safe Policy Optimization for Continuous Controlcited by 140
  7. 2020Learning to be Safe: Deep RL with a Safety Criticcited by 109
  8. 2020Conservative Safety Critics for Explorationcited by 107
  9. 2020Offline Meta-Reinforcement Learning with Advantage Weightingcited by 92
  10. 2017Leave no Trace: Learning to Reset for Safe and Autonomous Reinforcement Learningcited by 86
  11. 2018Evolved Policy Gradientscited by 86
  12. 2015Safe Controller Optimization for Quadrotors with Gaussian Processescited by 77
  13. 2019Meta reinforcement learning as task inferencecited by 75
  14. 2020Decoupling Exploration and Exploitation for Meta-Reinforcement Learning without Sacrificescited by 59
  15. 2019Meta Reinforcement Learning for Sim-to-real Domain Adaptationcited by 49
  16. 2019Robust Model Predictive Shielding for Safe Reinforcement Learning with Stochastic Dynamicscited by 45
  17. 2018Meta Reinforcement Learning with Latent Variable Gaussian Processescited by 44
  18. 2020Rapidly Adaptable Legged Robots via Evolutionary Meta-Learningcited by 42
  19. 2020Cautious Adaptation For Reinforcement Learning in Safety-Critical Settingscited by 42
  20. 2020ABC-LMPC: Safe Sample-Based Learning MPC for Stochastic Nonlinear Dynamical Systems with Adjustable Boundary Conditionscited by 8
  21. 2020FOCAL: Efficient Fully-Offline Meta-Reinforcement Learning via Distance Metric Learning and Behavior Regularizationcited by 6
  22. 2020Safe Reinforcement Learning through Meta-learned Instinctscited by 4
  23. 2015Safe Platooning of Unmanned Aerial Vehicles via Reachabilitycited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 0; 2025: 1; 2026: 3.

7 papers.

  1. 2023Safe Deep Policy Adaptationcited by 9
  2. 2023Reinforcement Learning by Guided Safe Explorationcited by 5
  3. 2023Safe Reinforcement Learning with Dead-Ends Avoidance and Recoverycited by 4
  4. 2025Safe In-Context Reinforcement Learningcited by 2
  5. 2026WINFlowNets: Warm-up Integrated Networks Training of Generative Flow Networks for Robotics and Machine Fault Adaptationcited by 0
  6. 2026Why Does Agentic Safety Fail to Generalize Across Tasks?cited by 0
  7. 2026Latent Q-Barrier Shielding for Safe In-Context Reinforcement Learningcited by 0