# Risk-Aware Safe Reinforcement Learning for Control of Stochastic Linear Systems

arXiv: https://arxiv.org/abs/2505.09734 · HTML: https://mlsanity.com/paper/2505.09734/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Safe and constrained RL, one of 15 directions in Deep reinforcement learning methods.
- Builds on 19 of its 45 references that are in the map.
- First posted on arXiv 14 May 2025.

## Placed in

- Deep reinforcement learning methods / [Safe and constrained RL](https://mlsanity.com/deeprl/safe-and-constrained-rl/)

## Builds on (most cited first, 19 of 19)

- [Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learning](https://mlsanity.com/paper/2108.06266/) (2021)
- [End-to-End Safe Reinforcement Learning through Barrier Functions for Safety-Critical Continuous Control Tasks](https://mlsanity.com/paper/1903.08792/) (2019)
- [Control Barrier Function Based Quadratic Programs for Safety Critical Systems](https://mlsanity.com/paper/1609.06408/) (2016)
- [Safe Reinforcement Learning in Constrained Markov Decision Processes](https://mlsanity.com/paper/2008.06626/) (2020)
- [Safe Reinforcement Learning Using Robust MPC](https://mlsanity.com/paper/1906.04005/) (2019)
- [Safety-Critical Model Predictive Control with Discrete-Time Control Barrier Function](https://mlsanity.com/paper/2007.11718/) (2020)
- [Robust Model Predictive Shielding for Safe Reinforcement Learning with Stochastic Dynamics](https://mlsanity.com/paper/1910.10885/) (2019)
- [Multi-Layered Safety for Legged Robots via Control Barrier Functions and Model Predictive Control](https://mlsanity.com/paper/2011.00032/) (2020)
- [Online Shielding for Reinforcement Learning](https://mlsanity.com/paper/2212.01861/) (2022)
- [Safety-Aware Optimal Control of Stochastic Systems Using Conditional Value-at-Risk](https://mlsanity.com/paper/1802.07903/) (2018)
- [Low-complexity Learning of Linear Quadratic Regulators from Noisy Data](https://mlsanity.com/paper/2005.01082/) (2020)
- [Data-driven distributionally robust LQR with multiplicative noise](https://mlsanity.com/paper/1912.09990/) (2019)
- [Safe Control in the Presence of Stochastic Uncertainties](https://mlsanity.com/paper/2104.01259/) (2021)
- [Self-triggered Control for Safety Critical Systems using Control Barrier Functions](https://mlsanity.com/paper/1903.03692/) (2019)
- [Controller design for robust invariance from noisy data](https://mlsanity.com/paper/2007.13181/) (2020)
- [Data-driven Safe Control of Linear Systems Under Epistemic and Aleatory Uncertainties](https://mlsanity.com/paper/2202.04495/) (2022)
- [Data-based guarantees of set invariance properties](https://mlsanity.com/paper/1911.12293/) (2019)
- [Control Barrier Function based Quadratic Programs Introduce Undesirable Asymptotically Stable Equilibria](https://mlsanity.com/paper/2003.07819/) (2020)
- [On Direct vs Indirect Data-Driven Predictive Control](https://mlsanity.com/paper/2103.14936/) (2021)

## Abstract (arXiv)

This paper presents a risk-aware safe reinforcement learning (RL) control design for stochastic discrete-time linear systems. Rather than using a safety certifier to myopically intervene with the RL controller, a risk-informed safe controller is also learned besides the RL controller, and the RL and safe controllers are combined together. Several advantages come along with this approach: 1) High-confidence safety can be certified without relying on a high-fidelity system model and using limited data available, 2) Myopic interventions and convergence to an undesired equilibrium can be avoided by deciding on the contribution of two stabilizing controllers, and 3) highly efficient and computationally tractable solutions can be provided by optimizing over a scalar decision variable and linear programming polyhedral sets. To learn safe controllers with a large invariant set, piecewise affine controllers are learned instead of linear controllers. To this end, the closed-loop system is first represented using collected data, a decision variable, and noise. The effect of the decision variable on the variance of the safe violation of the closed-loop system is formalized. The decision variable is then designed such that the probability of safety violation for the learned closed-loop system is minimized. It is shown that this control-oriented approach reduces the data requirements and can also reduce the variance of safety violations. Finally, to integrate the safe and RL controllers, a new data-driven interpolation technique is introduced. This method aims to maintain the RL agent's optimal implementation while ensuring its safety within environments characterized by noise. The study concludes with a simulation example that serves to validate the theoretical results.