mlsanityData to 14 Oct 2026

Online Shielding for Reinforcement Learning

arXiv 2212.01861 ↗First posted 4 Dec 2022cs.LG cs.LOcited by 22

Not on the map. Why?

Abstract (arXiv)

Besides the recent impressive results on reinforcement learning (RL), safety is still one of the major research challenges in RL. RL is a machine-learning approach to determine near-optimal policies in Markov decision processes (MDPs). In this paper, we consider the setting where the safety-relevant fragment of the MDP together with a temporal logic safety specification is given and many safety violations can be avoided by planning ahead a short time into the future. We propose an approach for online safety shielding of RL agents. During runtime, the shield analyses the safety of each available action. For any action, the shield computes the maximal probability to not violate the safety specification within the next k steps when executing this action. Based on this probability and a given threshold, the shield decides whether to block an action from the agent. Existing offline shielding approaches compute exhaustively the safety of all state-action combinations ahead of time, resulting in huge computation times and large memory consumption. The intuition behind online shielding is to compute at runtime the set of all states that could be reached in the near future. For each of these states, the safety of all available actions is analysed and used for shielding as soon as one of the considered states is reached. Our approach is well suited for high-level planning problems where the time between decisions can be used for safety computations and it is sustainable for the agent to wait until these computations are finished. For our evaluation, we selected a 2-player version of the classical computer game SNAKE. The game represents a high-level planning problem that requires fast decisions and the multiplayer setting induces a large state space, which is computationally expensive to analyse exhaustively.

Key facts. Cited by 22 papers in the map (data through 14 Oct 2026). Builds on 22 of its 46 references that are in the map. First posted on arXiv 4 Dec 2022.

Builds on

22 of 46 references.

References · 22

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  2. 2016Concrete Problems in AI Safetycited by 1,642
  3. 2017Safe Reinforcement Learning via Shieldingcited by 449
  4. 2018A Lyapunov-based Approach to Safe Reinforcement Learningcited by 297
  5. 2019End-to-End Safe Reinforcement Learning through Barrier Functions for Safety-Critical Continuous Control Taskscited by 289
  6. 2018Omega-Regular Objectives in Model-Free Reinforcement Learningcited by 75
  7. 2021Safe Multi-Agent Reinforcement Learning via Shieldingcited by 61
  8. 2019Robust Model Predictive Shielding for Safe Reinforcement Learning with Stochastic Dynamicscited by 47
  9. 2020Cautious Reinforcement Learning with Logical Constraintscited by 45
  10. 2015Shield Synthesis: Runtime Enforcement for Reactive Systemscited by 42
  11. 2019Learning Robotic Manipulation through Visual Planning and Actingcited by 41
  12. 2019Certified Reinforcement Learning with Logic Guidancecited by 36
10 more
  1. 2017A storm is Coming: A Modern Probabilistic Model Checkercited by 34
  2. 2020Verifiably Safe Exploration for End-to-End Reinforcement Learningcited by 34
  3. 2019MAMPS: Safe Multi-Agent Reinforcement Learning via Model Predictive Shieldingcited by 20
  4. 2018Barrier-Certified Adaptive Reinforcement Learning with Applications to Brushbot Navigationcited by 18
  5. 2020Adaptive Shielding under Uncertaintycited by 18
  6. 2019Verifiably Safe Off-Model Reinforcement Learningcited by 16
  7. 2021Shielding Atari Games with Bounded Presciencecited by 14
  8. 2021TEMPEST -- Synthesis Tool for Reactive Systems and Shields in Probabilistic Environmentscited by 13
  9. 2020Online Shielding for Stochastic Systemscited by 11
  10. 2019L*-Based Learning of Markov Decision Processes (Extended Version)cited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 4; 2025: 10; 2026: 6.

22 papers.

Sort:

By area:

  1. 2024Dynamic Model Predictive Shielding for Provably Safe Reinforcement Learningcited by 15
  2. 2023SMARLA: A Safety Monitoring Approach for Deep Reinforcement Learning Agentscited by 10
  3. 2024Online Safety Analysis for LLMs: a Benchmark, an Assessment, and a Path Forwardcited by 6
  4. 2024Safe Reinforcement Learning in Black-Box Environments via Adaptive Shieldingcited by 6
  5. 2026Breaking Up with Normatively Monolithic Agency with GRACE: A Reason-Based Neuro-Symbolic Architecture for Safe and Ethical AI Alignmentcited by 5
  6. 2024CommonPower: A Framework for Safe Data-Driven Smart Grid Controlcited by 4
  7. 2025Conformal Safety Shielding for Imperfect-Perception Agentscited by 3
  8. 2025HypRL: Reinforcement Learning of Control Policies for Hyperpropertiescited by 2
  9. 2025Automaton Constrained Q-Learningcited by 2
  10. 2026Shields to Guarantee Probabilistic Safety in MDPscited by 2
  11. 2026Robust Shielding for Safe Reinforcement Learningcited by 2
  12. 2025SPoRt -- Safe Policy Ratio: Certified Training and Deployment of Task Policies in Model-Free RLcited by 1
10 more
  1. 2025Follow the STARs: Dynamic ω-Regular Shielding of Learned Policiescited by 1
  2. 2026Conformal Signal Temporal Logic for Robust Reinforcement Learning Control: A Case Studycited by 1
  3. 2023Constrained Exploration in Reinforcement Learning with Optimality Preservationcited by 0
  4. 2025Human-aligned Safe Reinforcement Learning for Highway On-Ramp Merging in Dense Trafficcited by 0
  5. 2025Risk-Aware Safe Reinforcement Learning for Control of Stochastic Linear Systemscited by 0
  6. 2025Efficient Dynamic Shielding for Parametric Safety Specificationscited by 0
  7. 2025SORA-ATMAS: Adaptive Trust Management and Multi-LLM Aligned Governance for Future Smart Citiescited by 0
  8. 2025Adaptive GR(1) Specification Repair for Liveness-Preserving Shielding in Reinforcement Learningcited by 0
  9. 2026COOL-MC: Verifying and Explaining RL Policies for Platelet Inventory Managementcited by 0
  10. 2026Data-Driven Synthesis of Probabilistic Controlled Invariant Sets for Linear MDPscited by 0