mlsanityData to 14 Oct 2026

A Dynamical Systems Framework for Reinforcement Learning Safety and Robustness Verification

arXiv 2508.15588 ↗First posted 21 Aug 2025cs.AIcited by 0

On the map: Deep reinforcement learning methods → Safe and constrained RL

Abstract (arXiv)

The application of reinforcement learning to safety-critical systems is limited by the lack of formal methods for verifying the robustness and safety of learned policies. This paper introduces a novel framework that addresses this gap by analyzing the combination of an RL agent and its environment as a discrete-time autonomous dynamical system. By leveraging tools from dynamical systems theory, specifically the Finite-Time Lyapunov Exponent (FTLE), we identify and visualize Lagrangian Coherent Structures (LCS) that act as the hidden "skeleton" governing the system's behavior. We demonstrate that repelling LCS function as safety barriers around unsafe regions, while attracting LCS reveal the system's convergence properties and potential failure modes, such as unintended "trap" states. To move beyond qualitative visualization, we introduce a suite of quantitative metrics, Mean Boundary Repulsion (MBR), Aggregated Spurious Attractor Strength (ASAS), and Temporally-Aware Spurious Attractor Strength (TASAS), to formally measure a policy's safety margin and robustness. We further provide a method for deriving local stability guarantees and extend the analysis to handle model uncertainty. Through experiments in both discrete and continuous control environments, we show that this framework provides a comprehensive and interpretable assessment of policy behavior, successfully identifying critical flaws in policies that appear successful based on reward alone.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Safe and constrained RL, one of 15 directions in Deep reinforcement learning methods. Builds on 25 of its 38 references that are in the map. First posted on arXiv 21 Aug 2025.

Builds on

25 of 38 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Verified Safe Reinforcement Learning for Neural Network Dynamic Models25 May

Other references · 24

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  3. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  4. 2017Towards A Rigorous Science of Interpretable Machine Learningcited by 1,416
  5. 2017Constrained Policy Optimizationcited by 983
  6. 2019Solving Rubik's Cube with a Robot Handcited by 734
  7. 2020Deep Reinforcement Learning for Autonomous Driving: A Surveycited by 721
  8. 2017A Brief Survey of Deep Reinforcement Learningcited by 483
  9. 2017Robust Adversarial Reinforcement Learningcited by 451
  10. 2016Safe, Multi-Agent, Reinforcement Learning for Autonomous Drivingcited by 311
  11. 2018A Lyapunov-based Approach to Safe Reinforcement Learningcited by 297
  12. 2020Robust Deep Reinforcement Learning against Adversarial Perturbations on State Observationscited by 238
12 more
  1. 2022A Survey on Offline Reinforcement Learning: Taxonomy, Review, and Open Problemscited by 231
  2. 2021Robust Reinforcement Learning on State Observations with Learned Optimal Adversarycited by 135
  3. 2019Online Robustness Training for Deep Reinforcement Learningcited by 26
  4. 2024SINDy-RL: Interpretable and Efficient Model-Based Reinforcement Learningcited by 26
  5. 2024Robust Deep Reinforcement Learning Through Adversarial Attacks and Training : A Surveycited by 13
  6. 2023Safe Reinforcement Learning with Dual Robustnesscited by 10
  7. 2023Finite Time Lyapunov Exponent Analysis of Model Predictive Control and Reinforcement Learningcited by 7
  8. 2022Verified Probabilistic Policies for Deep Reinforcement Learningcited by 6
  9. 2024Enhancing Robustness in Deep Reinforcement Learning: A Lyapunov Exponent Approachcited by 4
  10. 2019Reinforcement Learning for Learning of Dynamical Systems in Uncertain Environment: a Tutorialcited by 2
  11. 2022Review of Metrics to Measure the Stability, Robustness and Resilience of Reinforcement Learningcited by 2
  12. 2024Adversarial Attacks on Reinforcement Learning Agents for Command and Controlcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Learning Control Policies to Provably Satisfy Hard Affine Constraints for Black-Box Hybrid Dynamical Systems
  2. 2024Sampling-based Safe Reinforcement Learning for Nonlinear Dynamical Systems
  3. 2026Dyna-Style Safety Augmented Reinforcement Learning: Staying Safe in the Face of Uncertainty
  4. 2026CAPSULE: Control-Theoretic Action Perturbations for Safe Uncertainty-Aware Reinforcement Learning
  5. 2024Learning safety critics via a non-contractive binary bellman operator
  6. 2026Sampling-Based Safe Reinforcement Learning
  7. 2026Safe Continual Reinforcement Learning in Non-stationary Environments
  8. 2025Safe and Near-Optimal Control with Online Dynamics Learning