mlsanityData to 14 Oct 2026

Data-Driven Synthesis of Probabilistic Controlled Invariant Sets for Linear MDPs

arXiv 2604.02727 ↗First posted 3 Apr 2026eess.SY cs.SYcited by 0

On the map: Deep reinforcement learning methods → Safe and constrained RL

Abstract (arXiv)

We study data-driven computation of probabilistic controlled invariant sets (PCIS) for safety-critical reinforcement learning under unknown dynamics. Assuming a linear MDP model, we use regularized least squares and self-normalized confidence bounds to construct a conservative estimate of the states from which the system can be kept inside a prescribed safe region over an (N)-step horizon, together with the corresponding set-valued safe action map. This construction is obtained through a backward recursion and can be interpreted as a conservative approximation of the (N)-step safety predecessor operator. When the associated conservative-inclusion event holds, a conservative fixed point of the approximate recursion can be certified as an ((N,ε))-PCIS with confidence at least (η). For continuous state spaces, we introduce a lattice abstraction and a Lipschitz-based discretization error bound to obtain a tractable approximation scheme. Finally, we use the resulting conservative fixed-point approximation as a runtime candidate PCIS in a practical shielding architecture with iterative updates, and illustrate the approach on a numerical experiment.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Safe and constrained RL, one of 15 directions in Deep reinforcement learning methods. Builds on 33 of its 48 references that are in the map. First posted on arXiv 3 Apr 2026.

Builds on

33 of 48 references.

References · 33

  1. 2017Constrained Policy Optimizationcited by 983
  2. 2019Provably Efficient Reinforcement Learning with Linear Function Approximationcited by 520
  3. 2017Safe Reinforcement Learning via Shieldingcited by 449
  4. 2021Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learningcited by 371
  5. 2017Safe Model-based Reinforcement Learning with Stability Guaranteescited by 366
  6. 2018A Lyapunov-based Approach to Safe Reinforcement Learningcited by 297
  7. 2020Nearly Minimax Optimal Reinforcement Learning for Linear Mixture Markov Decision Processescited by 200
  8. 2022A Review of Safe Reinforcement Learning: Methods, Theory and Applicationscited by 181
  9. 2020Recovery RL: Safe Reinforcement Learning with Learned Recovery Zonescited by 175
  10. 2017A General Safety Framework for Learning-Based Control in Uncertain Robotic Systemscited by 163
  11. 2018A predictive safety filter for learning-based control of constrained nonlinear dynamical systemscited by 118
  12. 2020Safe Reinforcement Learning in Constrained Markov Decision Processescited by 115
21 more
  1. 2023The Safety Filter: A Unified View of Safety-Critical Control in Autonomous Systemscited by 88
  2. 2019Learning for Safety-Critical Control with Control Barrier Functionscited by 69
  3. 2022Reachability Constrained Reinforcement Learningcited by 59
  4. 2022Making Linear MDPs Practical via Contrastive Representation Learningcited by 52
  5. 2021Safe Reinforcement Learning with Linear Function Approximationcited by 35
  6. 2022Provably Efficient Model-Free Constrained RL with Linear Function Approximationcited by 31
  7. 2023Safe Exploration in Reinforcement Learning: A Generalized Formulation and Algorithmscited by 28
  8. 2022Online Shielding for Reinforcement Learningcited by 22
  9. 2020Control Barrier Functions for Unknown Nonlinear Systems using Gaussian Processescited by 20
  10. 2021Reinforcement Learning in Linear MDPs: Constant Regret and Representation Selectioncited by 18
  11. 2023A Near-Optimal Algorithm for Safe Reinforcement Learning Under Instantaneous Hard Constraintscited by 17
  12. 2020Learning-based Symbolic Abstractions for Nonlinear Control Systemscited by 11
  13. 2023Safe Reinforcement Learning with Instantaneous Constraints: The Role of Aggressive Explorationcited by 7
  14. 2019Computing Probabilistic Controlled Invariant Setscited by 6
  15. 2023Approximate Model-Based Shielding for Safe Reinforcement Learningcited by 5
  16. 2019Computing controlled invariant sets from data using convex optimizationcited by 3
  17. 2019Compositional Abstraction-based Synthesis for Networks of Stochastic Switched Systemscited by 1
  18. 2021Data-driven synthesis of Robust Invariant Sets and Controllerscited by 1
  19. 2023Data-Driven Computation of Robust Invariant Sets and Gain-Scheduled Controllers for Linear Parameter-Varying Systemscited by 1
  20. 2023Data-driven invariant set for nonlinear systems with application to command governorscited by 1
  21. 2024Controlled Invariant Sets for Gaussian Process State Space Modelscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Provably Safe, Yet Scalable Reinforcement Learning
  2. 2025Probabilistic Shielding for Safe Reinforcement Learning
  3. 2026CAPSULE: Control-Theoretic Action Perturbations for Safe Uncertainty-Aware Reinforcement Learning
  4. 2024Sampling-based Safe Reinforcement Learning for Nonlinear Dynamical Systems
  5. 2025Safe exploration in reproducing kernel Hilbert spaces
  6. 2026Sampling-Based Safe Reinforcement Learning
  7. 2026Robust Shielding for Safe Reinforcement Learning
  8. 2026Robust Koopman Control Barrier Filters for Safe Actor-Critic Reinforcement Learning