mlsanityData to 14 Oct 2026

Skill-based Safe Reinforcement Learning with Risk Planning

arXiv 2505.01619 ↗First posted 2 May 2025cs.LG cs.AIcited by 0

On the map: Deep reinforcement learning methods → Safe and constrained RL

Abstract (arXiv)

Safe Reinforcement Learning (Safe RL) aims to ensure safety when an RL agent conducts learning by interacting with real-world environments where improper actions can induce high costs or lead to severe consequences. In this paper, we propose a novel Safe Skill Planning (SSkP) approach to enhance effective safe RL by exploiting auxiliary offline demonstration data. SSkP involves a two-stage process. First, we employ PU learning to learn a skill risk predictor from the offline demonstration data. Then, based on the learned skill risk predictor, we develop a novel risk planning process to enhance online safe RL and learn a risk-averse safe policy efficiently through interactions with the online RL environment, while simultaneously adapting the skill risk predictor to the environment. We conduct experiments in several benchmark robotic simulation environments. The experimental results demonstrate that the proposed approach consistently outperforms previous state-of-the-art safe RL methods.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Safe and constrained RL, one of 15 directions in Deep reinforcement learning methods. Builds on 17 of its 31 references that are in the map. First posted on arXiv 2 May 2025.

Builds on

17 of 31 references.

References · 17

  1. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  2. 2016Generative Adversarial Imitation Learningcited by 1,597
  3. 2017Constrained Policy Optimizationcited by 983
  4. 2018Reward Constrained Policy Optimizationcited by 417
  5. 2021Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learningcited by 371
  6. 2020Accelerating Reinforcement Learning with Learned Skill Priorscited by 223
  7. 2017Positive-Unlabeled Learning with Non-Negative Risk Estimatorcited by 219
  8. 2020Recovery RL: Safe Reinforcement Learning with Learned Recovery Zonescited by 175
  9. 2021Constraints Penalized Q-learning for Safe Offline Reinforcement Learningcited by 91
  10. 2022Safe Reinforcement Learning by Imagining the Near Futurecited by 67
  11. 2023Datasets and Benchmarks for Offline Safe Reinforcement Learningcited by 59
  12. 2016Class-prior Estimation for Learning from Positive and Unlabeled Datacited by 46
5 more
  1. 2019Positive-Unlabeled Reward Learningcited by 37
  2. 2020Safe Reinforcement Learning for Autonomous Vehicles through Parallel Constrained Policy Optimizationcited by 35
  3. 2019The Differentiable Cross-Entropy Methodcited by 27
  4. 2016Estimating the class prior and posterior from noisy positives and unlabeled datacited by 25
  5. 2022SAFER: Data-Efficient and Safe Reinforcement Learning via Skill Acquisitioncited by 6

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Safe Reinforcement Learning with Minimal Supervision
  2. 2022A Review of Safe Reinforcement Learning: Methods, Theory and Applications
  3. 2024A Dynamic Safety Shield for Safe and Efficient Reinforcement Learning of Navigation Tasks
  4. 2023Safe Reinforcement Learning in a Simulated Robotic Arm
  5. 2026Safe-RULE: Safe Reinforcement UnLEarning
  6. 2025Safety Modulation: Enhancing Safety in Reinforcement Learning through Cost-Modulated Rewards
  7. 2026Turning Safety into Competence: Minimally Exploitable Robot Policies via Safety-Filtered Reinforcement Learning
  8. 2024Safety Optimized Reinforcement Learning via Multi-Objective Policy Optimization