mlsanityData to 14 Oct 2026

Approximate Model-Based Shielding for Safe Reinforcement Learning

arXiv 2308.00707 ↗First posted 27 Jul 2023cs.LG cs.AI cs.SYcited by 5

Not on the map. Why?

Abstract (arXiv)

Reinforcement learning (RL) has shown great potential for solving complex tasks in a variety of domains. However, applying RL to safety-critical systems in the real-world is not easy as many algorithms are sample-inefficient and maximising the standard RL objective comes with no guarantees on worst-case performance. In this paper we propose approximate model-based shielding (AMBS), a principled look-ahead shielding algorithm for verifying the performance of learned RL policies w.r.t. a set of given safety constraints. Our algorithm differs from other shielding approaches in that it does not require prior knowledge of the safety-relevant dynamics of the system. We provide a strong theoretical justification for AMBS and demonstrate superior performance to other safety-aware approaches on a set of Atari games with state-dependent safety-labels.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 30 of its 44 references that are in the map. First posted on arXiv 27 Jul 2023.

Builds on

30 of 44 references.

References · 30

  1. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  2. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,653
  3. 2016Concrete Problems in AI Safetycited by 1,642
  4. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,264
  5. 2023Mastering Diverse Domains through World Modelscited by 1,253
  6. 2020Mastering Atari with Discrete World Modelscited by 1,007
  7. 2017Constrained Policy Optimizationcited by 983
  8. 2018DeepMind Control Suitecited by 956
  9. 2017Rainbow: Combining Improvements in Deep Reinforcement Learningcited by 786
  10. 2018Recurrent World Models Facilitate Policy Evolutioncited by 785
  11. 2019When to Trust Your Model: Model-Based Policy Optimizationcited by 773
  12. 2017Safe Reinforcement Learning via Shieldingcited by 449
18 more
  1. 2018Implicit Quantile Networks for Distributional Reinforcement Learningcited by 369
  2. 2017Revisiting the Arcade Learning Environment: Evaluation Protocols and Open Problems for General Agentscited by 269
  3. 2020Projection-Based Constrained Policy Optimizationcited by 203
  4. 2019Lyapunov-based Safe Policy Optimization for Continuous Controlcited by 145
  5. 2019IPO: Interior-point Policy Optimization under Constraintscited by 132
  6. 2018Dopamine: A Research Framework for Deep Reinforcement Learningcited by 122
  7. 2020A Game Theoretic Framework for Model Based Reinforcement Learningcited by 73
  8. 2022Safe Reinforcement Learning by Imagining the Near Futurecited by 67
  9. 2022Constrained Policy Optimization via Bayesian World Modelscited by 57
  10. 2015Shield Synthesis: Runtime Enforcement for Reactive Systemscited by 42
  11. 2021Learning Barrier Certificates: Towards Safe Reinforcement Learning with Zero Training-time Violationscited by 39
  12. 2019The MineRL 2019 Competition on Sample Efficient Reinforcement Learning using Human Priorscited by 26
  13. 2019Learning Belief Representations for Imitation Learning in POMDPscited by 24
  14. 2023Learning in POMDPs is Sample-Efficient with Hindsight Observabilitycited by 18
  15. 2021Shielding Atari Games with Bounded Presciencecited by 14
  16. 2023Model-based Dynamic Shielding for Safe and Efficient Multi-Agent Reinforcement Learningcited by 11
  17. 2021Do Androids Dream of Electric Fences? Safety-Aware Reinforcement Learning with Latent Shieldingcited by 9
  18. 2023Approximate Shielding of Atari Agents for Safe Explorationcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 2; 2026: 2.

5 papers.

  1. 2024Safe Reinforcement Learning in Black-Box Environments via Adaptive Shieldingcited by 6
  2. 2025Predictive Safety Shield for Dyna-Q Reinforcement Learningcited by 1
  3. 2025Adaptive GR(1) Specification Repair for Liveness-Preserving Shielding in Reinforcement Learningcited by 0
  4. 2026Data-Driven Synthesis of Probabilistic Controlled Invariant Sets for Linear MDPscited by 0
  5. 2026Adaptive Probabilistic Shielding by Learning MDPs for Safe Reinforcement Learningcited by 0