mlsanityData to 14 Oct 2026

ProSh: Probabilistic Shielding for Model-free Reinforcement Learning

arXiv 2510.15720 ↗First posted 17 Oct 2025cs.LG cs.AIcited by 0

On the map: Deep reinforcement learning methods → Safe and constrained RL

Abstract (arXiv)

Safety is a major concern in reinforcement learning (RL): we aim at developing RL systems that not only perform optimally, but are also safe to deploy by providing formal guarantees about their safety. To this end, we introduce Probabilistic Shielding via Risk Augmentation (ProSh), a model-free algorithm for safe reinforcement learning under cost constraints. ProSh augments the Constrained MDP state space with a risk budget and enforces safety by applying a shield to the agent's policy distribution using a learned cost critic. The shield ensures that all sampled actions remain safe in expectation. We also show that optimality is preserved when the environment is deterministic. Since ProSh is model-free, safety during training depends on the knowledge we have acquired about the environment. We provide a tight upper-bound on the cost in expectation, depending only on the backup-critic accuracy, that is always satisfied during training. Under mild, practically achievable assumptions, ProSh guarantees safety even at training time, as shown in the experiments.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Safe and constrained RL, one of 15 directions in Deep reinforcement learning methods. Builds on 20 of its 39 references that are in the map. First posted on arXiv 17 Oct 2025.

Builds on

20 of 39 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Leveraging Approximate Model-based Shielding for Probabilistic Safety Guarantees in Continuous Environments1 Feb

Other references · 19

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  3. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  4. 2015Trust Region Policy Optimizationcited by 2,580
  5. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  6. 2015Dueling Network Architectures for Deep Reinforcement Learningcited by 785
  7. 2020Responsive Safety in Reinforcement Learning by PID Lagrangian Methodscited by 316
  8. 2018A Lyapunov-based Approach to Safe Reinforcement Learningcited by 297
  9. 2018Learning to Drive in a Daycited by 273
  10. 2020FLAMBE: Structural Complexity and Representation Learning of Low Rank MDPscited by 233
  11. 2019Lyapunov-based Safe Policy Optimization for Continuous Controlcited by 145
  12. 2023Safety-Gymnasium: A Unified Safe Reinforcement Learning Benchmarkcited by 143
7 more
  1. 2019IPO: Interior-point Policy Optimization under Constraintscited by 132
  2. 2020Learning to be Safe: Deep RL with a Safety Criticcited by 116
  3. 2021Safe Multi-Agent Reinforcement Learning via Shieldingcited by 61
  4. 2023Safe Reinforcement Learning via Probabilistic Logic Shieldscited by 42
  5. 2021State Augmented Constrained Reinforcement Learning: Overcoming the Limitations of Learning with Rewardscited by 40
  6. 2020Constrained Markov Decision Processes via Backward Value Functionscited by 33
  7. 2020WD3: Taming the Estimation Bias in Deep Reinforcement Learningcited by 16

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Probabilistic Shielding for Safe Reinforcement Learning
  2. 2026Robust Probabilistic Shielding for Safe Offline Reinforcement Learning
  3. 2026Adaptive Probabilistic Shielding by Learning MDPs for Safe Reinforcement Learning
  4. 2025Adaptive Shielding for Safe Reinforcement Learning under Hidden-Parameter Dynamics Shifts
  5. 2026Robust Shielding for Safe Reinforcement Learning
  6. 2024Safe Reinforcement Learning in Black-Box Environments via Adaptive Shielding
  7. 2026Scenario Generation for Risk-Aware Reinforcement Learning with Probably Approximately Safe Guarantees
  8. 2025Provably Optimal Reinforcement Learning under Safety Filtering