mlsanityData to 7 Oct 2026

Constrained Meta-Reinforcement Learning for Adaptable Safety Guarantee with Differentiable Convex Programming

arXiv 2312.10230 ↗First posted 15 Dec 2023cs.AI cs.LGcited by 8

Not on the map. Why?

Abstract (arXiv)

Despite remarkable achievements in artificial intelligence, the deployability of learning-enabled systems in high-stakes real-world environments still faces persistent challenges. For example, in safety-critical domains like autonomous driving, robotic manipulation, and healthcare, it is crucial not only to achieve high performance but also to comply with given constraints. Furthermore, adaptability becomes paramount in non-stationary domains, where environmental parameters are subject to change. While safety and adaptability are recognized as key qualities for the new generation of AI, current approaches have not demonstrated effective adaptable performance in constrained settings. Hence, this paper breaks new ground by studying the unique challenges of ensuring safety in non-stationary environments by solving constrained problems through the lens of the meta-learning approach (learning-to-learn). While unconstrained meta-learning al-ready encounters complexities in end-to-end differentiation of the loss due to the bi-level nature, its constrained counterpart introduces an additional layer of difficulty, since the constraints imposed on task-level updates complicate the differentiation process. To address the issue, we first employ successive convex-constrained policy updates across multiple tasks with differentiable convexprogramming, which allows meta-learning in constrained scenarios by enabling end-to-end differentiation. This approach empowers the agent to rapidly adapt to new tasks under non-stationarity while ensuring compliance with safety constraints.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 26 of its 42 references that are in the map. First posted on arXiv 15 Dec 2023.

Builds on

26 of 42 references.

References · 26

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  3. 2015Trust Region Policy Optimizationcited by 2,503
  4. 2019Meta-Learning with Implicit Gradientscited by 523
  5. 2019Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variablescited by 372
  6. 2017Safe Model-based Reinforcement Learning with Stability Guaranteescited by 347
  7. 2021Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learningcited by 337
  8. 2015Risk-Constrained Reinforcement Learning with Percentile Risk Criteriacited by 320
  9. 2020Responsive Safety in Reinforcement Learning by PID Lagrangian Methodscited by 302
  10. 2018A Lyapunov-based Approach to Safe Reinforcement Learningcited by 278
  11. 2016Control Barrier Function Based Quadratic Programs for Safety Critical Systemscited by 261
  12. 2019End-to-End Safe Reinforcement Learning through Barrier Functions for Safety-Critical Continuous Control Taskscited by 256
14 more
  1. 2020Robust Deep Reinforcement Learning against Adversarial Perturbations on State Observationscited by 235
  2. 2020Projection-Based Constrained Policy Optimizationcited by 195
  3. 2019IPO: Interior-point Policy Optimization under Constraintscited by 119
  4. 2022Neural-Fly Enables Rapid Learning for Agile Flight in Strong Windscited by 80
  5. 2020Reinforcement Learning for Safety-Critical Control under Model Uncertainty, using Control Lyapunov Functions and Control Barrier Functionscited by 61
  6. 2021Sample-Efficient Learning of Stackelberg Equilibria in General-Sum Gamescited by 40
  7. 2019Policy Optimization Provably Converges to Nash Equilibria in Zero-Sum Linear Quadratic Gamescited by 38
  8. 2021Safe Pontryagin Differentiable Programmingcited by 25
  9. 2021Meta-Adaptive Nonlinear Control: Theory and Algorithmscited by 23
  10. 2020Safe Reinforcement Learning of Control-Affine Systems with Vertex Networkscited by 8
  11. 2020FISAR: Forward Invariant Safe Reinforcement Learning with a Deep Neural Network-Based Optimizecited by 5
  12. 2021Temporal-Logic-Based Intermittent, Optimal, and Safe Continuous-Time Learning for Trajectory Trackingcited by 2
  13. 2023Wasserstein Distributionally Robust Control Barrier Function using Conditional Value-at-Risk with Differentiable Convex Programmingcited by 2
  14. 2023Distributionally Safe Reinforcement Learning under Model Uncertainty: A Single-Level Approach by Differentiable Convex Programmingcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 2; 2026: 3.

8 papers.

  1. 2025One Subgoal at a Time: Zero-Shot Generalization to Arbitrary Linear Temporal Logic Requirements in Multi-Task Reinforcement Learningcited by 8
  2. 2024Out-of-Distribution Adaptation in Offline RL: Counterfactual Reasoning via Causal Normalizing Flowscited by 2
  3. 2024Hierarchical Meta-Reinforcement Learning via Automated Macro-Action Discoverycited by 2
  4. 2024Sparsity-based Safety Conservatism for Constrained Offline Reinforcement Learningcited by 1
  5. 2026Constrained Meta Reinforcement Learning with Provable Test-Time Safetycited by 1
  6. 2025Context-aware Constrained Reinforcement Learning Based Energy-Efficient Power Scheduling for Non-stationary XR Data Trafficcited by 0
  7. 2026Safe Continual Reinforcement Learning in Non-stationary Environmentscited by 0
  8. 2026Safe Meta-Reinforcement Learning via Information Space Reachabilitycited by 0