mlsanityData to 14 Oct 2026

FoSeRL: Formal Sequential Robustness Certification for Reinforcement Learning Policies

arXiv 2610.04754 ↗First posted 3 Oct 2026cs.LGcited by 0

On the map: Deep reinforcement learning methods → Safe and constrained RL

Abstract (arXiv)

Even a few action perturbations can substantially degrade the performance of a deployed decision policy. Certifying the resulting return loss is challenging in stochastic environments, where returns vary even without an attack. We introduce FoSeRL, a framework for certifying deployed RL policies against precommitted, temporally sparse action attacks. The deployed policy is unchanged, with no smoothing or retraining. Certification requires a resettable simulator supporting shared randomness and independent one-step successor queries, but no analytical dynamics model. FoSeRL certifies that an attacked episode loses no more than a prescribed amount of return relative to the same episode unattacked, with at least a target probability and at a user-specified confidence level. Both runs share the initial state and randomness, so the measured loss reflects the attack, not the episode; carrying the running return gap as a state coordinate makes it the terminal value, reducing trajectory-level certification to terminal safety. Time-dependent barrier conditions on the augmented state bound the terminal failure probability: satisfied exactly, they certify every admissible precommitted attack; learned from sampled trajectories and verified on held-out data, they certify the same guarantee under a specified attack-episode setting. Across six stochastic continuous-control environments and three RL policy families (TD3, SAC, and PPO), FoSeRL certifies non-trivial cardinality--magnitude robustness frontiers, achieves substantially larger certified budgets than policy smoothing, and reveals marked robustness differences among policies with comparable nominal performance.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Safe and constrained RL, one of 15 directions in Deep reinforcement learning methods. Builds on 26 of its 47 references that are in the map. First posted on arXiv 3 Oct 2026.

Builds on

26 of 47 references.

References · 26

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  3. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  4. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,920
  5. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,809
  6. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,241
  7. 2020MOPO: Model-based Offline Policy Optimizationcited by 636
  8. 2020MOReL : Model-Based Offline Reinforcement Learningcited by 590
  9. 2017Robust Adversarial Reinforcement Learningcited by 451
  10. 2017Adversarial Attacks on Neural Network Policiescited by 320
  11. 2019Information-Theoretic Considerations in Batch Reinforcement Learningcited by 287
  12. 2020Robust Deep Reinforcement Learning against Adversarial Perturbations on State Observationscited by 238
14 more
  1. 2019Action Robust Reinforcement Learning and Applications in Continuous Controlcited by 174
  2. 2017Tactics of Adversarial Attack on Deep Reinforcement Learning Agentscited by 166
  3. 2021Robust Reinforcement Learning on State Observations with Learned Optimal Adversarycited by 135
  4. 2020Robust Deep Reinforcement Learning through Adversarial Losscited by 75
  5. 2020Stealthy and Efficient Adversarial Attacks against Deep Reinforcement Learningcited by 63
  6. 2022Efficient Adversarial Training without Attacking: Worst-Case-Aware Robust Reinforcement Learningcited by 43
  7. 2021CROP: Certifying Robust Policies for Reinforcement Learning through Functional Smoothingcited by 42
  8. 2021Policy Smoothing for Provably Robust Reinforcement Learningcited by 40
  9. 2021Learning Barrier Certificates: Towards Safe Reinforcement Learning with Zero Training-time Violationscited by 39
  10. 2022COPA: Certifying Robust Policies for Offline Reinforcement Learning against Poisoning Attackscited by 26
  11. 2019Spatiotemporally Constrained Action Space Attacks on Deep Reinforcement Learning Agentscited by 25
  12. 2020Certifiable Robustness to Adversarial State Uncertainty in Deep Reinforcement Learningcited by 24
  13. 2023Reward Certification for Policy Smoothed Reinforcement Learningcited by 7
  14. 2023Certifiably Robust Reinforcement Learning through Model-Based Abstract Interpretationcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Verification-Guided Falsification for Safe RL via Explainable Abstraction and Risk-Aware Exploration
  2. 2026CORE-RL: Confidence-Oriented Reliability Evaluation of Black-Box Reinforcement Learning Policies
  3. 2025Action Robust Reinforcement Learning via Optimal Adversary Aware Policy Optimization
  4. 2026Provably Safe, Yet Scalable Reinforcement Learning
  5. 2025Learning Robust Penetration Testing Policies under Partial Observability: A systematic evaluation
  6. 2026A Survey on the Verification of Reinforcement Learning Policies
  7. 2026Conformal Signal Temporal Logic for Robust Reinforcement Learning Control: A Case Study
  8. 2026Robust Probabilistic Shielding for Safe Offline Reinforcement Learning