# Learning Predictive Safety Filter via Decomposition of Robust Invariant Set

arXiv: https://arxiv.org/abs/2311.06769 · HTML: https://mlsanity.com/paper/2311.06769/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 46 references that are in the map.
- First posted on arXiv 12 Nov 2023.

## Builds on (most cited first, 30 of 31)

- [Constrained Policy Optimization](https://mlsanity.com/paper/1705.10528/) (2017)
- [Control Barrier Functions: Theory and Applications](https://mlsanity.com/paper/1903.11199/) (2019)
- [Risk-Constrained Reinforcement Learning with Percentile Risk Criteria](https://mlsanity.com/paper/1512.01629/) (2015)
- [Control Barrier Function Based Quadratic Programs for Safety Critical Systems](https://mlsanity.com/paper/1609.06408/) (2016)
- [Hamilton-Jacobi Reachability: A Brief Overview and Recent Advances](https://mlsanity.com/paper/1709.07523/) (2017)
- [A General Safety Framework for Learning-Based Control in Uncertain Robotic Systems](https://mlsanity.com/paper/1705.01292/) (2017)
- [Learning-based Model Predictive Control for Safe Exploration](https://mlsanity.com/paper/1803.08287/) (2018)
- [A predictive safety filter for learning-based control of constrained nonlinear dynamical systems](https://mlsanity.com/paper/1812.05506/) (2018)
- [Safe Nonlinear Control Using Robust Neural Lyapunov-Barrier Functions](https://mlsanity.com/paper/2109.06697/) (2021)
- [Learning Safe Multi-Agent Control with Decentralized Neural Barrier Certificates](https://mlsanity.com/paper/2101.05436/) (2021)
- [Safe Reinforcement Learning Using Robust Control Barrier Functions](https://mlsanity.com/paper/2110.05415/) (2021)
- [Reachability Constrained Reinforcement Learning](https://mlsanity.com/paper/2205.07536/) (2022)
- [State-wise Safe Reinforcement Learning: A Survey](https://mlsanity.com/paper/2302.03122/) (2023)
- [Safety and Liveness Guarantees through Reach-Avoid Reinforcement Learning](https://mlsanity.com/paper/2112.12288/) (2021)
- [On the Robustness of Safe Reinforcement Learning under Observational Perturbations](https://mlsanity.com/paper/2205.14691/) (2022)
- [Linear model predictive safety certification for learning-based control](https://mlsanity.com/paper/1803.08552/) (2018)
- [Safe Control Under Input Limits with Neural Control Barrier Functions](https://mlsanity.com/paper/2211.11056/) (2022)
- [Joint Synthesis of Safety Certificate and Safe Control Policy using Constrained Reinforcement Learning](https://mlsanity.com/paper/2111.07695/) (2021)
- [FaSTrack: a Modular Framework for Real-Time Motion Planning and Guaranteed Safe Tracking](https://mlsanity.com/paper/2102.07039/) (2021)
- [A computationally efficient robust model predictive control framework for uncertain nonlinear systems -- extended version](https://mlsanity.com/paper/1910.12081/) (2019)
- [Persistently Feasible Robust Safe Control by Safety Index Synthesis and Convex Semi-Infinite Programming](https://mlsanity.com/paper/2209.06896/) (2022)
- [Robust Control Barrier Functions with Sector-Bounded Uncertainties](https://mlsanity.com/paper/2109.02537/) (2021)
- [Robust Control Barrier Functions under High Relative Degree and Input Constraints for Satellite Trajectories](https://mlsanity.com/paper/2107.04094/) (2021)
- [Predictive safety filter using system level synthesis](https://mlsanity.com/paper/2212.02111/) (2022)
- [Robust Nonlinear Optimal Control via System Level Synthesis](https://mlsanity.com/paper/2301.04943/) (2023)
- [Robust Model Predictive Control with Polytopic Model Uncertainty through System Level Synthesis](https://mlsanity.com/paper/2203.11375/) (2022)
- [Robust Safe Reinforcement Learning under Adversarial Disturbances](https://mlsanity.com/paper/2310.07207/) (2023)
- [Disturbance Observer-based Robust Control Barrier Functions](https://mlsanity.com/paper/2203.12855/) (2022)
- [Computationally Efficient Robust Model Predictive Control for Uncertain System Using Causal State-Feedback Parameterization](https://mlsanity.com/paper/2208.08431/) (2022)
- [Robust Optimal Control for Nonlinear Systems with Parametric Uncertainties via System Level Synthesis](https://mlsanity.com/paper/2304.00752/) (2023)

## Built on by (most cited first, 3 of 3)

- [Implicit Safe Set Algorithm for Provably Safe Reinforcement Learning](https://mlsanity.com/paper/2405.02754/) (2024)
- [Provably Safe, Yet Scalable Reinforcement Learning](https://mlsanity.com/paper/2606.14536/) (2026)
- [Absolute State-wise Constrained Policy Optimization: High-Probability State-wise Constraints Satisfaction](https://mlsanity.com/paper/2410.01212/) (2024)

## Abstract (arXiv)

Ensuring safety of nonlinear systems under model uncertainty and external disturbances is crucial, especially for real-world control tasks. Predictive methods such as robust model predictive control (RMPC) require solving nonconvex optimization problems online, which leads to high computational burden and poor scalability. Reinforcement learning (RL) works well with complex systems, but pays the price of losing rigorous safety guarantee. This paper presents a theoretical framework that bridges the advantages of both RMPC and RL to synthesize safety filters for nonlinear systems with state- and action-dependent uncertainty. We decompose the robust invariant set (RIS) into two parts: a target set that aligns with terminal region design of RMPC, and a reach-avoid set that accounts for the rest of RIS. We propose a policy iteration approach for robust reach-avoid problems and establish its monotone convergence. This method sets the stage for an adversarial actor-critic deep RL algorithm, which simultaneously synthesizes a reach-avoid policy network, a disturbance policy network, and a reach-avoid value network. The learned reach-avoid policy network is utilized to generate nominal trajectories for online verification, which filters potentially unsafe actions that may drive the system into unsafe regions when worst-case disturbances are applied. We formulate a second-order cone programming (SOCP) approach for online verification using system level synthesis, which optimizes for the worst-case reach-avoid value of any possible trajectories. The proposed safety filter requires much lower computational complexity than RMPC and still enjoys persistent robust safety guarantee. The effectiveness of our method is illustrated through a numerical example.