Guiding Safe Exploration with Weakest Preconditions
Not on the map. Why?
Abstract (arXiv)
In reinforcement learning for safety-critical settings, it is often desirable for the agent to obey safety constraints at all points in time, including during training. We present a novel neurosymbolic approach called SPICE to solve this safe exploration problem. SPICE uses an online shielding layer based on symbolic weakest preconditions to achieve a more precise safety analysis than existing tools without unduly impacting the training process. We evaluate the approach on a suite of continuous control benchmarks and show that it can achieve comparable performance to existing safe learning techniques while incurring fewer safety violations. Additionally, we present theoretical results showing that SPICE converges to the optimal safe policy under reasonable assumptions.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 21 of its 35 references that are in the map. First posted on arXiv 28 Sep 2022.
Builds on
21 of 35 references.
References · 21
- 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
- 2018Soft Actor-Critic Algorithms and Applicationscited by 1,179
- 2017Constrained Policy Optimizationcited by 928
- 2019When to Trust Your Model: Model-Based Policy Optimizationcited by 753
- 2017Safe Reinforcement Learning via Shieldingcited by 427
- 2017Safe Model-based Reinforcement Learning with Stability Guaranteescited by 347
- 2018A Lyapunov-based Approach to Safe Reinforcement Learningcited by 278
- 2019End-to-End Safe Reinforcement Learning through Barrier Functions for Safety-Critical Continuous Control Taskscited by 256
- 2018Safe Exploration in Continuous Action Spacescited by 237
- 2020Projection-Based Constrained Policy Optimizationcited by 195
- 2018Verifiable Reinforcement Learning via Policy Extractioncited by 176
- 2017A General Safety Framework for Learning-Based Control in Uncertain Robotic Systemscited by 150
9 more
- 2018Learning-based Model Predictive Control for Safe Explorationcited by 116
- 2020Conservative Safety Critics for Explorationcited by 107
- 2022Reachability Constrained Reinforcement Learningcited by 52
- 2020Neurosymbolic Reinforcement Learning with Formally Verified Explorationcited by 49
- 2019An Inductive Synthesis Framework for Verifiable Reinforcement Learningcited by 39
- 2020Constrained Markov Decision Processes via Backward Value Functionscited by 33
- 2019Verifiably Safe Off-Model Reinforcement Learningcited by 16
- 2021Conservative and Adaptive Penalty for Model-Based Safe Reinforcement Learningcited by 13
- 2021Safe Reinforcement Learning Using Robust Action Governorcited by 12
Built on by
Citing papers per year: 2024: 2; 2025: 0; 2026: 1.
3 papers.