# Learning-Based Shielding for Safe Autonomy under Unknown Dynamics

arXiv: https://arxiv.org/abs/2410.07359 · HTML: https://mlsanity.com/paper/2410.07359/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Safe and constrained RL, one of 15 directions in Deep reinforcement learning methods.
- Builds on 14 of its 28 references that are in the map.
- First posted on arXiv 7 Oct 2024.

## Placed in

- Deep reinforcement learning methods / [Safe and constrained RL](https://mlsanity.com/deeprl/safe-and-constrained-rl/)

## Builds on (most cited first, 14 of 14)

- [Safe Reinforcement Learning via Shielding](https://mlsanity.com/paper/1708.08611/) (2017)
- [On Kernelized Multi-armed Bandits](https://mlsanity.com/paper/1704.00445/) (2017)
- [The Promises and Pitfalls of Deep Kernel Learning](https://mlsanity.com/paper/2102.12108/) (2021)
- [Shield Synthesis: Runtime Enforcement for Reactive Systems](https://mlsanity.com/paper/1501.02573/) (2015)
- [Safe Reinforcement Learning via Probabilistic Logic Shields](https://mlsanity.com/paper/2303.03226/) (2023)
- [Formal Synthesis of Stochastic Systems via Control Barrier Certificates](https://mlsanity.com/paper/1905.04585/) (2019)
- [A Barrier Function Approach to Finite-Time Stochastic System Verification and Control](https://mlsanity.com/paper/1909.05109/) (2019)
- [Formal Control Synthesis for Stochastic Neural Network Dynamic Models](https://mlsanity.com/paper/2203.05903/) (2022)
- [Error Bounds For Gaussian Process Regression Under Bounded Support Noise With Applications To Safety Certification](https://mlsanity.com/paper/2408.09033/) (2024)
- [Formal Verification of Unknown Dynamical Systems via Gaussian Process Regression](https://mlsanity.com/paper/2201.00655/) (2021)
- [Promises of Deep Kernel Learning for Control Synthesis](https://mlsanity.com/paper/2309.06569/) (2023)
- [Data-Driven Permissible Safe Control with Barrier Certificates](https://mlsanity.com/paper/2405.00136/) (2024)
- [Formal Abstraction of General Stochastic Systems via Noise Partitioning](https://mlsanity.com/paper/2309.10702/) (2023)
- [Shielded Deep Reinforcement Learning for Complex Spacecraft Tasking](https://mlsanity.com/paper/2403.05693/) (2024)

## Built on by (most cited first, 3 of 3)

- [Robust Shielding for Safe Reinforcement Learning](https://mlsanity.com/paper/2606.00270/) (2026)
- [Kernel-Based Safe Exploration in Deep Reinforcement Learning](https://mlsanity.com/paper/2605.22207/) (2026)
- [Adaptive Probabilistic Shielding by Learning MDPs for Safe Reinforcement Learning](https://mlsanity.com/paper/2608.19836/) (2026)

## Abstract (arXiv)

Shielding is a common method used to guarantee the safety of a system under a black-box controller, such as a neural network controller from deep reinforcement learning (DRL), with simpler, verified controllers. Existing shielding methods rely on formal verification through Markov Decision Processes (MDPs), assuming either known or finite-state models, which limits their applicability to DRL settings with unknown, continuous-state systems. This paper addresses these limitations by proposing a data-driven shielding methodology that guarantees safety for unknown systems under black-box controllers. The approach leverages Deep Kernel Learning to model the systems' one-step evolution with uncertainty quantification and constructs a finite-state abstraction as an Interval MDP (IMDP). By focusing on safety properties expressed in safe linear temporal logic (safe LTL), we develop an algorithm that computes the maximally permissive set of safe policies on the IMDP, ensuring avoidance of unsafe states. The algorithms soundness and computational complexity are demonstrated through theoretical proofs and experiments on nonlinear systems, including a high-dimensional autonomous spacecraft scenario.