mlsanityData to 14 Oct 2026

Safe Reinforcement Learning via Probabilistic Logic Shields

arXiv 2303.03226 ↗First posted 6 Mar 2023cs.AI cs.LGcited by 42

Not on the map. Why?

Cited by 21 in Deep reinforcement learning methods, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Safe Reinforcement learning (Safe RL) aims at learning optimal policies while staying safe. A popular solution to Safe RL is shielding, which uses a logical safety specification to prevent an RL agent from taking unsafe actions. However, traditional shielding techniques are difficult to integrate with continuous, end-to-end deep RL methods. To this end, we introduce Probabilistic Logic Policy Gradient (PLPG). PLPG is a model-based Safe RL technique that uses probabilistic logic programming to model logical safety constraints as differentiable functions. Therefore, PLPG can be seamlessly applied to any policy gradient algorithm while still providing the same convergence guarantees. In our experiments, we show that PLPG learns safer and more rewarding policies compared to other state-of-the-art shielding techniques.

Key facts. Cited by 42 papers in the map (data through 14 Oct 2026). Builds on 28 of its 77 references that are in the map. First posted on arXiv 6 Mar 2023.

Builds on

28 of 77 references.

References · 28

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2015Trust Region Policy Optimizationcited by 2,580
  3. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,258
  4. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 2,026
  5. 2017Constrained Policy Optimizationcited by 983
  6. 2017Safe Reinforcement Learning via Shieldingcited by 449
  7. 2018Reward Constrained Policy Optimizationcited by 417
  8. 2019End-to-End Safe Reinforcement Learning through Barrier Functions for Safety-Critical Continuous Control Taskscited by 289
  9. 2017A Semantic Loss Function for Deep Learning with Symbolic Knowledgecited by 270
  10. 2018Safe Exploration in Continuous Action Spacescited by 247
  11. 2018Verifiable Reinforcement Learning via Policy Extractioncited by 177
  12. 2022Semantic Probabilistic Layers for Neuro-Symbolic Learningcited by 103
16 more
  1. 2013A Differential Approach to Inference in Bayesian Networkscited by 93
  2. 2019Reinforcement Learning for Temporal Logic Control Synthesis with Probabilistic Satisfaction Guaranteescited by 77
  3. 2018Foundations for Restraining Bolts: Reinforcement Learning with LTLf/LDLf restraining specificationscited by 72
  4. 2017OptLayer - Practical Constrained Optimization for Deep Reinforcement Learning in the Real Worldcited by 70
  5. 2022Safe Reinforcement Learning via Shielding under Partial Observabilitycited by 56
  6. 2017QMDP-Net: Deep Learning for Planning under Partial Observabilitycited by 51
  7. 2020Neurosymbolic Reinforcement Learning with Formally Verified Explorationcited by 49
  8. 2019Robust Model Predictive Shielding for Safe Reinforcement Learning with Stochastic Dynamicscited by 47
  9. 2021Self-Supervised Online Reward Shaping in Sparse-Reward Environmentscited by 37
  10. 2020Verifiably Safe Exploration for End-to-End Reinforcement Learningcited by 34
  11. 2020Temporal-Logic-Based Reward Shaping for Continuing Reinforcement Learning Taskscited by 28
  12. 2020Deep Inverse Q-learning with Constraintscited by 17
  13. 2020Enforcing Almost-Sure Reachability in POMDPscited by 14
  14. 2021Shielding Atari Games with Bounded Presciencecited by 14
  15. 2021Reinforcement Learning with External Knowledge by using Logical Neural Networkscited by 10
  16. 2013Parameterized Verification of Asynchronous Shared-Memory Systemscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 13; 2025: 14; 2026: 13.

42 papers.

Sort:

By area:

  1. 2024KLay: Accelerating Arithmetic Circuits for Neurosymbolic AIcited by 16
  2. 2024Dynamic Model Predictive Shielding for Provably Safe Reinforcement Learningcited by 15
  3. 2025Probabilistic Shielding for Safe Reinforcement Learningcited by 12
  4. 2025Paradigms of AI Evaluation: Mapping Goals, Methodologies and Culturecited by 11
  5. 2024NeSyA: Neurosymbolic Automatacited by 10
  6. 2025Building Intelligent Agents with Neuro-Symbolic Conceptscited by 10
  7. 2024Compositional Shielding and Reinforcement Learning for Multi-Agent Systemscited by 8
  8. 2025Defining neurosymbolic AIcited by 8
  9. 2023Towards Safe Autonomous Driving Policies using a Neuro-Symbolic Deep Reinforcement Learning Approachcited by 7
  10. 2024Semirings for Probabilistic and Neuro-Symbolic Logic Programmingcited by 6
  11. 2024Safe Reinforcement Learning in Black-Box Environments via Adaptive Shieldingcited by 6
  12. 2024Leveraging Approximate Model-based Shielding for Probabilistic Safety Guarantees in Continuous Environmentscited by 5
30 more
  1. 2024Realizable Continuous-Space Shields for Safe Reinforcement Learningcited by 3
  2. 2024Learning-Based Shielding for Safe Autonomy under Unknown Dynamicscited by 3
  3. 2025Provably Optimal Reinforcement Learning under Safety Filteringcited by 3
  4. 2025The Oversight Game: Learning to Cooperatively Balance an AI Agent's Safety and Autonomycited by 3
  5. 2024Composing Reinforcement Learning Policies, with Formal Guaranteescited by 2
  6. 2024Test Where Decisions Matter: Importance-driven Testing for Deep Reinforcement Learningcited by 2
  7. 2025Adaptive Shielding for Safe Reinforcement Learning under Hidden-Parameter Dynamics Shiftscited by 2
  8. 2026Neuro-Symbolic Artificial Intelligence: A Task-Directed Survey in the Black-Box Models Eracited by 2
  9. 2026Shields to Guarantee Probabilistic Safety in MDPscited by 2
  10. 2026Robust Shielding for Safe Reinforcement Learningcited by 2
  11. 2023Integrating Symbolic Reasoning into Neural Generative Models for Design Generationcited by 1
  12. 2024Towards Interpretable Reinforcement Learning with Constrained Normalizing Flow Policiescited by 1
  13. 2024Think Smart, Act SMARL! Analyzing Probabilistic Logic Shields for Multi-Agent Reinforcement Learningcited by 1
  14. 2025Follow the STARs: Dynamic ω-Regular Shielding of Learned Policiescited by 1
  15. 2026Safety Generalization Under Distribution Shift in Safe Reinforcement Learning: A Diabetes Testbedcited by 1
  16. 2026Neuro-symbolic Action Masking for Deep Reinforcement Learningcited by 1
  17. 2026Position: A Three-Layer Probabilistic Assume-Guarantee Architecture Is Structurally Required for Safe LLM Agent Deploymentcited by 1
  18. 2026Latent Q-Barrier Shielding for Safe In-Context Reinforcement Learningcited by 1
  19. 2025Efficient Dynamic Shielding for Parametric Safety Specificationscited by 0
  20. 2025Safe Reinforcement Learning with a Predictive Safety Filter for Motion Planning and Control: A Drifting Vehicle Examplecited by 0
  21. 2025LexiCon: a Benchmark for Planning under Temporal Constraints in Natural Languagecited by 0
  22. 2025ProSh: Probabilistic Shielding for Model-free Reinforcement Learningcited by 0
  23. 2025Guardian: Decoupling Exploration from Safety in Reinforcement Learningcited by 0
  24. 2025Exploring Structures of Inferential Mechanisms through Simplistic Digital Circuitscited by 0
  25. 2026COOL-MC: Verifying and Explaining RL Policies for Platelet Inventory Managementcited by 0
  26. 2026Oracle-Guided Soft Shielding for Safe Move Prediction in Chesscited by 0
  27. 2026Energy Shields for Fairnesscited by 0
  28. 2026Neuro-Symbolic Injection of LTLf Constraints in Autoregressive Reinforcement Learning Policiescited by 0
  29. 2026Imagine to Ensure Safety in Hierarchical Reinforcement Learningcited by 0
  30. 2026PL-Guard: Probabilistic Logic Reasoning for LLM Guardrailscited by 0