# Control invariant set enhanced safe reinforcement learning: improved sampling efficiency, guaranteed stability and robustness

arXiv: https://arxiv.org/abs/2305.15602 · HTML: https://mlsanity.com/paper/2305.15602/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 41 references that are in the map.
- First posted on arXiv 24 May 2023.

## Builds on (most cited first, 17 of 17)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Safe Reinforcement Learning via Shielding](https://mlsanity.com/paper/1708.08611/) (2017)
- [Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learning](https://mlsanity.com/paper/2108.06266/) (2021)
- [Risk-Constrained Reinforcement Learning with Percentile Risk Criteria](https://mlsanity.com/paper/1512.01629/) (2015)
- [A Review of Safe Reinforcement Learning: Methods, Theory and Applications](https://mlsanity.com/paper/2205.10330/) (2022)
- [Reinforcement Learning for Selective Key Applications in Power Systems: Recent Advances and Future Challenges](https://mlsanity.com/paper/2102.01168/) (2021)
- [Robust Model Predictive Shielding for Safe Reinforcement Learning with Stochastic Dynamics](https://mlsanity.com/paper/1910.10885/) (2019)
- [Safe Reinforcement Learning via Projection on a Safe Set: How to Achieve Optimality?](https://mlsanity.com/paper/2004.00915/) (2020)
- [Model-based Constrained Reinforcement Learning using Generalized Control Barrier Function](https://mlsanity.com/paper/2103.01556/) (2021)
- [Computationally Efficient Safe Reinforcement Learning for Power Systems](https://mlsanity.com/paper/2110.10333/) (2021)
- [Practical Reinforcement Learning of Stabilizing Economic MPC](https://mlsanity.com/paper/1904.04614/) (2019)
- [Learning Region of Attraction for Nonlinear Systems](https://mlsanity.com/paper/2110.00731/) (2021)
- [Comparison between safety methods control barrier function vs. reachability analysis](https://mlsanity.com/paper/2106.13176/) (2021)
- [Computing Robust Controlled Invariant Sets of Linear Systems](https://mlsanity.com/paper/1601.00416/) (2016)
- [Robust economic model predictive control with zone tracking](https://mlsanity.com/paper/2109.09810/) (2021)
- [Computing robust control invariant sets of constrained nonlinear systems: A graph algorithm approach](https://mlsanity.com/paper/2009.13581/) (2020)
- [Control invariant set enhanced reinforcement learning for process control: improved sampling efficiency and guaranteed stability](https://mlsanity.com/paper/2304.05509/) (2023)

## Built on by (most cited first, 2 of 2)

- [Reinforcement Learning-based Control via Y-wise Affine Neural Networks (YANNs)](https://mlsanity.com/paper/2508.16474/) (2025)
- [Reinforcement Learning-based Control via Y-wise Affine Neural Networks: Comparative Case Studies for Chemical Processes](https://mlsanity.com/paper/2605.21211/) (2026)

## Abstract (arXiv)

Reinforcement learning (RL) is an area of significant research interest, and safe RL in particular is attracting attention due to its ability to handle safety-driven constraints that are crucial for real-world applications. This work proposes a novel approach to RL training, called control invariant set (CIS) enhanced RL, which leverages the advantages of utilizing the explicit form of CIS to improve stability guarantees and sampling efficiency. Furthermore, the robustness of the proposed approach is investigated in the presence of uncertainty. The approach consists of two learning stages: offline and online. In the offline stage, CIS is incorporated into the reward design, initial state sampling, and state reset procedures. This incorporation of CIS facilitates improved sampling efficiency during the offline training process. In the online stage, RL is retrained whenever the predicted next step state is outside of the CIS, which serves as a stability criterion, by introducing a Safety Supervisor to examine the safety of the action and make necessary corrections. The stability analysis is conducted for both cases, with and without uncertainty. To evaluate the proposed approach, we apply it to a simulated chemical reactor. The results show a significant improvement in sampling efficiency during offline training and closed-loop stability guarantee in the online implementation, with and without uncertainty.