# Robust Model Predictive Shielding for Safe Reinforcement Learning with Stochastic Dynamics

arXiv: https://arxiv.org/abs/1910.10885 · HTML: https://mlsanity.com/paper/1910.10885/

## Key facts

- Cited by 47 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Oct 2019.

## Built on by (most cited first, 30 of 47)

- [The Safety Filter: A Unified View of Safety-Critical Control in Autonomous Systems](https://mlsanity.com/paper/2309.05837/) (2023)
- [Enforcing Hard Constraints with Soft Barriers: Safe Reinforcement Learning in Unknown Stochastic Environments](https://mlsanity.com/paper/2209.15090/) (2022)
- [Safety and Liveness Guarantees through Reach-Avoid Reinforcement Learning](https://mlsanity.com/paper/2112.12288/) (2021)
- [Safe Reinforcement Learning Using Advantage-Based Intervention](https://mlsanity.com/paper/2106.09110/) (2021)
- [A Survey of Constraint Formulations in Safe Reinforcement Learning](https://mlsanity.com/paper/2402.02025/) (2024)
- [Safe Reinforcement Learning via Probabilistic Logic Shields](https://mlsanity.com/paper/2303.03226/) (2023)
- [Synthesis of Model Predictive Control and Reinforcement Learning: Survey and Classification](https://mlsanity.com/paper/2502.02133/) (2025)
- [Learning Control Policies for Stochastic Systems with Reach-avoid Guarantees](https://mlsanity.com/paper/2210.05308/) (2022)
- [Online Shielding for Reinforcement Learning](https://mlsanity.com/paper/2212.01861/) (2022)
- [CBF-RL: Safety Filtering Reinforcement Learning in Training with Control Barrier Functions](https://mlsanity.com/paper/2510.14959/) (2025)
- [Conservative and Adaptive Penalty for Model-Based Safe Reinforcement Learning](https://mlsanity.com/paper/2112.07701/) (2021)
- [Active Uncertainty Reduction for Safe and Efficient Interaction Planning: A Shielding-Aware Dual Control Approach](https://mlsanity.com/paper/2302.00171/) (2023)
- [Dynamic Model Predictive Shielding for Provably Safe Reinforcement Learning](https://mlsanity.com/paper/2405.13863/) (2024)
- [Shielding Atari Games with Bounded Prescience](https://mlsanity.com/paper/2101.08153/) (2021)
- [Spatial-Temporal-Aware Safe Multi-Agent Reinforcement Learning of Connected Autonomous Vehicles in Challenging Scenarios](https://mlsanity.com/paper/2210.02300/) (2022)
- [SHARP: Shielding-Aware Robust Planning for Safe and Efficient Human-Robot Interaction](https://mlsanity.com/paper/2110.00843/) (2021)
- [Model-based Dynamic Shielding for Safe and Efficient Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2304.06281/) (2023)
- [Do Androids Dream of Electric Fences? Safety-Aware Reinforcement Learning with Latent Shielding](https://mlsanity.com/paper/2112.11490/) (2021)
- [MESA: Offline Meta-RL for Safe Adaptation and Fault Tolerance](https://mlsanity.com/paper/2112.03575/) (2021)
- [Risk-Aware Reinforcement Learning through Optimal Transport Theory](https://mlsanity.com/paper/2309.06239/) (2023)
- [Learning Control Barrier Functions and their application in Reinforcement Learning: A Survey](https://mlsanity.com/paper/2404.16879/) (2024)
- [Towards Safe Autonomous Driving Policies using a Neuro-Symbolic Deep Reinforcement Learning Approach](https://mlsanity.com/paper/2307.01316/) (2023)
- [Scalable Synthesis of Verified Controllers in Deep Reinforcement Learning](https://mlsanity.com/paper/2104.10219/) (2021)
- [Uniformly Conservative Exploration in Reinforcement Learning](https://mlsanity.com/paper/2110.13060/) (2021)
- [Learning safety critics via a non-contractive binary bellman operator](https://mlsanity.com/paper/2401.12849/) (2024)
- [Safe Reinforcement Learning for Power System Control: A Review](https://mlsanity.com/paper/2407.00681/) (2024)
- [Control invariant set enhanced safe reinforcement learning: improved sampling efficiency, guaranteed stability and robustness](https://mlsanity.com/paper/2305.15602/) (2023)
- [Constraint-Guided Reinforcement Learning: Augmenting the Agent-Environment-Interaction](https://mlsanity.com/paper/2104.11918/) (2021)
- [Control invariant set enhanced reinforcement learning for process control: improved sampling efficiency and guaranteed stability](https://mlsanity.com/paper/2304.05509/) (2023)
- [Who Plays First? Optimizing the Order of Play in Stackelberg Games with Many Robots](https://mlsanity.com/paper/2402.09246/) (2024)

## Abstract (arXiv)

This paper proposes a framework for safe reinforcement learning that can handle stochastic nonlinear dynamical systems. We focus on the setting where the nominal dynamics are known, and are subject to additive stochastic disturbances with known distribution. Our goal is to ensure the safety of a control policy trained using reinforcement learning, e.g., in a simulated environment. We build on the idea of model predictive shielding (MPS), where a backup controller is used to override the learned policy as needed to ensure safety. The key challenge is how to compute a backup policy in the context of stochastic dynamics. We propose to use a tube-based robust NMPC controller as the backup controller. We estimate the tubes using sampled trajectories, leveraging ideas from statistical learning theory to obtain high-probability guarantees. We empirically demonstrate that our approach can ensure safety in stochastic systems, including cart-pole and a non-holonomic particle with random obstacles.