mlsanityData to 14 Oct 2026

On the Design of Safe Continual RL Methods for Control of Nonlinear Systems

arXiv 2502.15922 ↗First posted 21 Feb 2025cs.LG cs.ROcited by 2

On the map: Deep reinforcement learning methods → Safe and constrained RL

Abstract (arXiv)

Reinforcement learning (RL) algorithms have been successfully applied to control tasks associated with unmanned aerial vehicles and robotics. In recent years, safe RL has been proposed to allow the safe execution of RL algorithms in industrial and mission-critical systems that operate in closed loops. However, if the system operating conditions change, such as when an unknown fault occurs in the system, typical safe RL algorithms are unable to adapt while retaining past knowledge. Continual reinforcement learning algorithms have been proposed to address this issue. However, the impact of continual adaptation on the system's safety is an understudied problem. In this paper, we study the intersection of safe and continual RL. First, we empirically demonstrate that a popular continual RL algorithm, online elastic weight consolidation, is unable to satisfy safety constraints in non-linear systems subject to varying operating conditions. Specifically, we study the MuJoCo HalfCheetah and Ant environments with velocity constraints and sudden joint loss non-stationarity. Then, we show that an agent trained using constrained policy optimization, a safe RL algorithm, experiences catastrophic forgetting in continual learning settings. With this in mind, we explore a simple reward-shaping method to ensure that elastic weight consolidation prioritizes remembering both safety and task performance for safety-constrained, non-linear, and non-stationary dynamical systems.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Safe and constrained RL, one of 15 directions in Deep reinforcement learning methods. Builds on 17 of its 25 references that are in the map. First posted on arXiv 21 Feb 2025.

Builds on

17 of 25 references.

References · 17

  1. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  2. 2018Experience Replay for Continual Learningcited by 1,095
  3. 2017Constrained Policy Optimizationcited by 983
  4. 2017Safe Reinforcement Learning via Shieldingcited by 449
  5. 2017Safe Model-based Reinforcement Learning with Stability Guaranteescited by 366
  6. 2020Towards Continual Reinforcement Learning: A Review and Perspectivescited by 274
  7. 2022A Review of Safe Reinforcement Learning: Methods, Theory and Applicationscited by 181
  8. 2023Safety-Gymnasium: A Unified Safe Reinforcement Learning Benchmarkcited by 143
  9. 2019Safe Reinforcement Learning Using Robust MPCcited by 85
  10. 2018Safe Reinforcement Learning with Model Uncertainty Estimatescited by 72
  11. 2018Reinforcement Learning for UAV Attitude Controlcited by 48
  12. 2021Same State, Different Task: Continual Reinforcement Learning without Interferencecited by 43
5 more
  1. 2021Context-Aware Safe Reinforcement Learning for Non-Stationary Environmentscited by 33
  2. 2020Task-Agnostic Online Reinforcement Learning with an Infinite Mixture of Gaussian Processescited by 23
  3. 2015Safe Policy Search for Lifelong Reinforcement Learning with Sublinear Regretcited by 15
  4. 2022Dynamics-Adaptive Continual Reinforcement Learning via Progressive Contextualizationcited by 14
  5. 2023Sharing Lifelong Reinforcement Learning Knowledge via Modulating Maskscited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.

  1. 2025A Survey of Safe Reinforcement Learning and Constrained MDPs: A Technical Survey on Single-Agent and Multi-Agent Safetycited by 13
  2. 2026Safe Continual Reinforcement Learning in Non-stationary Environmentscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Safe Continual Reinforcement Learning under Nonstationarity via Adaptive Safety Constraints
  2. 2024Sampling-based Safe Reinforcement Learning for Nonlinear Dynamical Systems
  3. 2026Safe Continual Reinforcement Learning Methods for Nonstationary Environments. Towards a Survey of the State of the Art
  4. 2024Reinforcement Learning with Adaptive Regularization for Safe Control of Critical Systems
  5. 2024A Dynamic Safety Shield for Safe and Efficient Reinforcement Learning of Navigation Tasks
  6. 2025Advancements and Challenges in Continual Reinforcement Learning: A Comprehensive Review
  7. 2025Continual Reinforcement Learning for Cyber-Physical Systems: Lessons Learned and Open Challenges
  8. 2026SafeAdapt: Provably Safe Policy Updates in Deep Reinforcement Learning