mlsanityData to 7 Oct 2026

Robust Probabilistic Shielding for Safe Offline Reinforcement Learning

arXiv 2605.10293 ↗First posted 11 May 2026cs.LG cs.AIcited by 2

On the map: Offline RL and imitation learning from data → Safe offline RL

Abstract (arXiv)

In offline reinforcement learning (RL), we learn policies from fixed datasets without environment interaction. The major challenges are to provide guarantees on the (1) performance and (2) safety of the resulting policy. A technique called safe policy improvement (SPI) provides a performance guarantee: with high probability, the new policy outperforms a given baseline policy, which is assumed to be safe. Orthogonally, in the context of safe RL, a shield provides a safety guarantee by restricting the action space to those actions that are provably safe with respect to a given safety-relevant model. We integrate these paradigms by extending shielding to offline RL, relying solely on the available dataset and knowledge of safe and unsafe states. Then, we shield the policy improvement steps, guaranteeing, with high probability, a safe policy. Experimental results demonstrate that shielded SPI outperforms its unshielded counterpart, improving both average and worst-case performance, particularly in low-data regimes.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Safe offline RL, one of 13 directions in Offline RL and imitation learning from data. Builds on 17 of its 40 references that are in the map. First posted on arXiv 11 May 2026.

Builds on

17 of 40 references.

Closest ancestors · 3

Same direction, newest first

  1. 2023More for Less: Safe Policy Improvement With Stronger Performance Guarantees13 May
  2. 2021Multi-Objective SPIBB: Seldonian Offline Policy Improvement with Safety Constraints in Finite MDPs31 May
  3. 2019Safe Policy Improvement with an Estimated Baseline Policy11 Sep

Other references · 14

  1. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  2. 2017Safe Reinforcement Learning via Shieldingcited by 427
  3. 2017Safe Policy Improvement with Baseline Bootstrappingcited by 141
  4. 2016Safe Policy Improvement by Minimizing Robust Baseline Regretcited by 51
  5. 2017A storm is Coming: A Modern Probabilistic Model Checkercited by 34
  6. 2015Safety-Constrained Reinforcement Learning for MDPscited by 31
  7. 2020Towards Safe Policy Improvement for Non-Stationary MDPscited by 26
  8. 2020Adaptive Shielding under Uncertaintycited by 17
  9. 2015Permissive Controller Synthesis for Probabilistic Systemscited by 16
  10. 2025Probabilistic Shielding for Safe Reinforcement Learningcited by 12
  11. 2021Do Androids Dream of Electric Fences? Safety-Aware Reinforcement Learning with Latent Shieldingcited by 9
  12. 2022Automata Learning meets Shieldingcited by 9
2 more
  1. 2023Shielded Reinforcement Learning for Hybrid Systemscited by 8
  2. 2024Efficient Shield Synthesis via State-Space Transformationcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Adaptive Probabilistic Shielding by Learning MDPs for Safe Reinforcement Learningcited by 0
  2. 2026Protected Cores Are Not Enough: Certifying AI-Proposed Revisions of Temporal Specificationscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Datasets and Benchmarks for Offline Safe Reinforcement Learning
  2. 2022Safe Policy Improvement Approaches and their Limitations
  3. 2024Towards Fast Safe Online Reinforcement Learning via Policy Finetuning
  4. 2025Online Optimization for Offline Safe Reinforcement Learning
  5. 2025Feasibility-Aware Pessimistic Estimation: Toward Long-Horizon Safety in Offline RL
  6. 2024Safe Offline Reinforcement Learning with Feasibility-Guided Diffusion Model
  7. 2024Latent Safety-Constrained Policy Approach for Safe Offline Reinforcement Learning
  8. 2023Guided Online Distillation: Promoting Safe Reinforcement Learning by Offline Demonstration