# Probabilistic Safeguard for Reinforcement Learning Using Safety Index Guided Gaussian Process Models

arXiv: https://arxiv.org/abs/2210.01041 · HTML: https://mlsanity.com/paper/2210.01041/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Oct 2022.

## Built on by (most cited first, 15 of 15)

- [State-wise Safe Reinforcement Learning: A Survey](https://mlsanity.com/paper/2302.03122/) (2023)
- [GUARD: A Safe Reinforcement Learning Benchmark](https://mlsanity.com/paper/2305.13681/) (2023)
- [Verification of Neural Control Barrier Functions with Symbolic Derivative Bounds Propagation](https://mlsanity.com/paper/2410.16281/) (2024)
- [Gradient Shaping for Multi-Constraint Safe Reinforcement Learning](https://mlsanity.com/paper/2312.15127/) (2023)
- [State-wise Constrained Policy Optimization](https://mlsanity.com/paper/2306.12594/) (2023)
- [Safe Deep Policy Adaptation](https://mlsanity.com/paper/2310.08602/) (2023)
- [One Subgoal at a Time: Zero-Shot Generalization to Arbitrary Linear Temporal Logic Requirements in Multi-Task Reinforcement Learning](https://mlsanity.com/paper/2508.01561/) (2025)
- [Fully Bayesian Differential Gaussian Processes through Stochastic Differential Equations](https://mlsanity.com/paper/2408.06069/) (2024)
- [Implicit Safe Set Algorithm for Provably Safe Reinforcement Learning](https://mlsanity.com/paper/2405.02754/) (2024)
- [Absolute Policy Optimization](https://mlsanity.com/paper/2310.13230/) (2023)
- [Disentangling Uncertainties by Learning Compressed Data Representation](https://mlsanity.com/paper/2503.15801/) (2025)
- [A Survey of Reinforcement Learning-Based Motion Planning for Autonomous Driving: Lessons Learned from a Driving Task Perspective](https://mlsanity.com/paper/2503.23650/) (2025)
- [The Feasibility Theory of Constrained Reinforcement Learning: A Tutorial Study](https://mlsanity.com/paper/2404.10064/) (2024)
- [Absolute State-wise Constrained Policy Optimization: High-Probability State-wise Constraints Satisfaction](https://mlsanity.com/paper/2410.01212/) (2024)
- [Data-driven Control with Real-time Uncertainty Compensation for Multi-Fuel Engines](https://mlsanity.com/paper/2606.16171/) (2026)

## Abstract (arXiv)

Safety is one of the biggest concerns to applying reinforcement learning (RL) to the physical world. In its core part, it is challenging to ensure RL agents persistently satisfy a hard state constraint without white-box or black-box dynamics models. This paper presents an integrated model learning and safe control framework to safeguard any agent, where its dynamics are learned as Gaussian processes. The proposed theory provides (i) a novel method to construct an offline dataset for model learning that best achieves safety requirements; (ii) a parameterization rule for safety index to ensure the existence of safe control; (iii) a safety guarantee in terms of probabilistic forward invariance when the model is learned using the aforementioned dataset. Simulation results show that our framework guarantees almost zero safety violation on various continuous control tasks.