# Probabilistic Safety Guarantee for Stochastic Control Systems Using Average Reward MDPs

arXiv: https://arxiv.org/abs/2511.08419 · HTML: https://mlsanity.com/paper/2511.08419/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 22 references that are in the map.
- First posted on arXiv 11 Nov 2025.

## Builds on (most cited first, 12 of 12)

- [A General Safety Framework for Learning-Based Control in Uncertain Robotic Systems](https://mlsanity.com/paper/1705.01292/) (2017)
- [A predictive safety filter for learning-based control of constrained nonlinear dynamical systems](https://mlsanity.com/paper/1812.05506/) (2018)
- [Robust Optimization for Non-Convex Objectives](https://mlsanity.com/paper/1707.01047/) (2017)
- [Control Barrier Functions and Input-to-State Safety with Application to Automated Vehicles](https://mlsanity.com/paper/2206.03568/) (2022)
- [A Minimum Discounted Reward Hamilton-Jacobi Formulation for Computing Reachable Sets](https://mlsanity.com/paper/1809.00706/) (2018)
- [A Dynamic Safety Shield for Safe and Efficient Reinforcement Learning of Navigation Tasks](https://mlsanity.com/paper/2412.04153/) (2024)
- [Safe Exploration in Reinforcement Learning: Training Backup Control Barrier Functions with Zero Training Time Safety Violations](https://mlsanity.com/paper/2312.07828/) (2023)
- [Back to Base: Towards Hands-Off Learning via Safe Resets with Reach-Avoid Safety Filters](https://mlsanity.com/paper/2501.02620/) (2025)
- [On reachability of Markov chains: A long-run average approach](https://mlsanity.com/paper/1806.00052/) (2018)
- [Data-Driven and Stealthy Deactivation of Safety Filters](https://mlsanity.com/paper/2412.01346/) (2024)
- [Learning for Layered Safety-Critical Control with Predictive Control Barrier Functions](https://mlsanity.com/paper/2412.04658/) (2024)
- [Control Barrier Functions for Shared Control and Vehicle Safety](https://mlsanity.com/paper/2503.19994/) (2025)

## Abstract (arXiv)

Safety in stochastic control systems, which are subject to random noise with a known probability distribution, aims to compute policies that satisfy predefined operational constraints with high confidence throughout the uncertain evolution of the state variables. The unpredictable evolution of state variables poses a significant challenge for meeting predefined constraints using various control methods. To address this, we present a new algorithm that computes safe policies to determine the safety level across a finite state set. This algorithm reduces the safety objective to the standard average reward Markov Decision Process (MDP) objective. This reduction enables us to use standard techniques, such as linear programs, to compute and analyze safe policies. We validate the proposed method numerically on the Double Integrator and the Inverted Pendulum systems. Results indicate that the average-reward MDPs solution is more comprehensive, converges faster, and offers higher quality compared to the minimum discounted-reward solution.