Data to 14 Oct 2026

On the Robustness of Safe Reinforcement Learning under Observational Perturbations

arXiv 2205.14691 ↗First posted 29 May 2022cs.LG cs.AI cs.ROcited by 38

Not on the map. Why?

Abstract (arXiv)

Safe reinforcement learning (RL) trains a policy to maximize the task reward while satisfying safety constraints. While prior works focus on the performance optimality, we find that the optimal solutions of many safe RL problems are not robust and safe against carefully designed observational perturbations. We formally analyze the unique properties of designing effective observational adversarial attackers in the safe RL setting. We show that baseline adversarial attack techniques for standard RL tasks are not always effective for safe RL and propose two new approaches - one maximizes the cost and the other maximizes the reward. One interesting and counter-intuitive finding is that the maximum reward attack is strong, as it can both induce unsafe behaviors and make the attack stealthy by maintaining the reward. We further propose a robust training framework for safe RL and evaluate it via comprehensive experiments. This paper provides a pioneer work to investigate the safety and robustness of RL under observational attacks for future safe RL studies. Code is available at: urlhttps://github.com/liuzuxin/safe-rl-robustness

Key facts. Cited by 38 papers in the map (data through 14 Oct 2026). Builds on 57 of its 84 references that are in the map. First posted on arXiv 29 May 2022.

Builds on

57 of 84 references.

References · 57

2017Proximal Policy Optimization Algorithmscited by 15,215
2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
2015Continuous control with deep reinforcement learningcited by 3,407
2013Playing Atari with Deep Reinforcement Learningcited by 3,274
2016Improved Techniques for Training GANscited by 2,970
2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,498
2017Constrained Policy Optimizationcited by 983
2017Towards Principled Methods for Training Generative Adversarial Networkscited by 464
2018Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Surveycited by 460
2017Robust Adversarial Reinforcement Learningcited by 451
2017Safe Reinforcement Learning via Shieldingcited by 449

38 more

2018Reward Constrained Policy Optimizationcited by 417
2016Benchmarking Deep Reinforcement Learning for Continuous Controlcited by 371
2021Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learningcited by 371
2017Safe Model-based Reinforcement Learning with Stability Guaranteescited by 366
2015Risk-Constrained Reinforcement Learning with Percentile Risk Criteriacited by 337
2018On the Effectiveness of Interval Bound Propagation for Training Verifiably Robust Modelscited by 328
2017Adversarial Attacks on Neural Network Policiescited by 320
2020Responsive Safety in Reinforcement Learning by PID Lagrangian Methodscited by 316
2020Emergent Complexity and Zero-shot Transfer via Unsupervised Environment Designcited by 275
2016Safe and Efficient Off-Policy Reinforcement Learningcited by 264
2018Safe Exploration in Continuous Action Spacescited by 247
2020Robust Deep Reinforcement Learning against Adversarial Perturbations on State Observationscited by 238
2020Projection-Based Constrained Policy Optimizationcited by 203
2022A Review of Safe Reinforcement Learning: Methods, Theory and Applicationscited by 181
2021Maximum Entropy RL (Provably) Solves Some Robust RL Problemscited by 176
2020Recovery RL: Safe Reinforcement Learning with Learned Recovery Zonescited by 175
2019Action Robust Reinforcement Learning and Applications in Continuous Controlcited by 174
2017Tactics of Adversarial Attack on Deep Reinforcement Learning Agentscited by 166
2017Robust Deep Reinforcement Learning with Adversarial Attackscited by 154
2019Constrained Reinforcement Learning Has Zero Duality Gapcited by 151
2020Automatic Curriculum Learning For Deep RL: A Short Surveycited by 150
2019Lyapunov-based Safe Policy Optimization for Continuous Controlcited by 145
2021Robust Reinforcement Learning on State Observations with Learned Optimal Adversarycited by 135
2020Learning to be Safe: Deep RL with a Safety Criticcited by 116
2017Delving into adversarial attacks on deep policiescited by 104
2017Trial without Error: Towards Safe Reinforcement Learning via Human Interventioncited by 103
2018Reinforcement Learning with Perturbed Rewardscited by 79
2022Constrained Variational Policy Optimization for Safe Reinforcement Learningcited by 71
2022Saute RL: Almost Surely Safe Reinforcement Learning Using State Augmentationcited by 69
2018Accelerated Primal-Dual Policy Optimization for Safe Reinforcement Learningcited by 62
2020Adversarial Machine Learning in Image Classification: A Survey Towards the Defender's Perspectivecited by 59
2019Convergent Policy Optimization for Safe Reinforcement Learningcited by 57
2022Constrained Policy Optimization via Bayesian World Modelscited by 57
2021Who Is the Strongest Enemy? Towards Optimal and Efficient Evasion Attacks in Deep RLcited by 55
2019DAC: The Double Actor-Critic Architecture for Learning Optionscited by 43
2021Learning Barrier Certificates: Towards Safe Reinforcement Learning with Zero Training-time Violationscited by 39
2021Multi-Agent Constrained Policy Optimisationcited by 35
2020Model-based Adversarial Meta-Reinforcement Learningcited by 33

50 most cited of 57 All 57 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 3; 2023: 11; 2024: 10; 2025: 6; 2026: 8.

38 papers.

Sort:

By area:

2023Constrained Decision Transformer for Offline Safe Reinforcement Learningcited by 69
2022SafeBench: A Benchmarking Platform for Safety Evaluation of Autonomous Vehiclescited by 66
2023Datasets and Benchmarks for Offline Safe Reinforcement Learningcited by 59
2022Trustworthy Reinforcement Learning Against Intrinsic Vulnerabilities: Robustness, Safety, and Generalizabilitycited by 36
2023Safe Offline Reinforcement Learning with Real-Time Budget Constraintscited by 28
2023Constraint-Conditioned Policy Optimization for Versatile Safe Reinforcement Learningcited by 18
2023A Human-Centered Safe Robot Reinforcement Learning Framework with Interactive Behaviorscited by 15
2023Learning Shared Safety Constraints from Multi-task Demonstrationscited by 15
2023Guided Online Distillation: Promoting Safe Reinforcement Learning by Offline Demonstrationcited by 14
2025Robust Gymnasium: A Unified Modular Benchmark for Robust Reinforcement Learningcited by 13
2026Failure-Aware RL: Reliable Offline-to-Online Reinforcement Learning with Self-Recovery for Real-World Manipulationcited by 11
2023Safe Reinforcement Learning with Dual Robustnesscited by 10

26 more

2024Temporal Logic Specification-Conditioned Decision Transformer for Offline Safe Reinforcement Learningcited by 10
2024OASIS: Conditional Distribution Shaping for Offline Safe Reinforcement Learningcited by 10
2023Optimal Transport Perturbations for Safe Reinforcement Learning with Robustness Guaranteescited by 9
2024Enhancing Efficiency of Safe Reinforcement Learning via Sample Manipulationcited by 9
2022Targeted Adversarial Attacks on Deep Reinforcement Learning Policies via Model Checkingcited by 5
2024Towards Unraveling and Improving Generalization in World Modelscited by 4
2025A Survey of Reinforcement Learning for Optimization in Automationcited by 4
2023Learning Predictive Safety Filter via Decomposition of Robust Invariant Setcited by 3
2024Feasibility Consistent Representation Learning for Safe Reinforcement Learningcited by 3
2023Wasserstein Distributionally Robust Control Barrier Function using Conditional Value-at-Risk with Differentiable Convex Programmingcited by 2
2024On Minimizing Adversarial Counterfactual Error in Adversarial RLcited by 2
2025PNAct: Crafting Backdoor Attacks in Safe Reinforcement Learningcited by 2
2024Tolerance of Reinforcement Learning Controllers against Deviations in Cyber Physical Systemscited by 1
2024Bidirectional Emergent Language in Situated Environmentscited by 1
2025CAMP in the Odyssey: Provably Robust Reinforcement Learning with Certified Radius Maximizationcited by 1
2026Finite-State Controllers for (Hidden-Model) POMDPs using Deep Reinforcement Learningcited by 1
2026TCRL: Temporal-Coupled Adversarial Training for Robust Constrained Reinforcement Learning in Worst-Case Scenarioscited by 1
2024On the Perturbed States for Transformed Input-robust Reinforcement Learningcited by 0
2024Constrained Optimal Fuel Consumption of HEVs under Observational Noisecited by 0
2025TraCeS: Learning Per-Timestep Constraint-Violation Credit from Sparse Trajectory-Level Labelscited by 0
2025Off-Policy Actor-Critic for Adversarial Observation Robustness: Virtual Alternative Training via Symmetric Policy Evaluationcited by 0
2026Demonstration-Guided Observation Attacks on Black-Box Safe Reinforcement Learning Controllers for Robotic Systemscited by 0
2026Transferable Reinforcement Learning via Probabilistic Latent Embeddings and Dynamic Policy Adaptation for Sim-to-Real Deploymentcited by 0
2026Safe-RULE: Safe Reinforcement UnLEarningcited by 0
2026SIGMA: Structured Noise-Effect-Aware Grouped Multi-Agent Aggregationcited by 0
2026Q-learning Penalized Transformer for Safe Offline Reinforcement Learningcited by 0