Data to 14 Oct 2026

Efficient Adversarial Training without Attacking: Worst-Case-Aware Robust Reinforcement Learning

arXiv 2210.05927 ↗First posted 12 Oct 2022cs.LG cs.AI cs.CRcited by 43

Not on the map. Why?

Cited by 22 in Deep reinforcement learning methods, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Recent studies reveal that a well-trained deep reinforcement learning (RL) policy can be particularly vulnerable to adversarial perturbations on input observations. Therefore, it is crucial to train RL agents that are robust against any attacks with a bounded budget. Existing robust training methods in deep RL either treat correlated steps separately, ignoring the robustness of long-term rewards, or train the agents and RL-based attacker together, doubling the computational burden and sample complexity of the training process. In this work, we propose a strong and efficient robust training framework for RL, named Worst-case-aware Robust RL (WocaR-RL) that directly estimates and optimizes the worst-case reward of a policy under bounded l_p attacks without requiring extra samples for learning an attacker. Experiments on multiple environments show that WocaR-RL achieves state-of-the-art performance under various strong attacks, and obtains significantly higher training efficiency than prior state-of-the-art robust training methods. The code of this work is available at https://github.com/umd-huang-lab/WocaR-RL.

Key facts. Cited by 43 papers in the map (data through 14 Oct 2026). Builds on 45 of its 57 references that are in the map. First posted on arXiv 12 Oct 2022.

Builds on

45 of 57 references.

References · 45

2017Proximal Policy Optimization Algorithmscited by 15,215
2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
2015Continuous control with deep reinforcement learningcited by 3,407
2013Playing Atari with Deep Reinforcement Learningcited by 3,274
2015Trust Region Policy Optimizationcited by 2,580
2015Deep Reinforcement Learning with Double Q-learningcited by 1,745
2019Certified Adversarial Robustness via Randomized Smoothingcited by 1,340
2015Prioritized Experience Replaycited by 1,099
2015Dueling Network Architectures for Deep Reinforcement Learningcited by 785
2017Provable defenses against adversarial examples via the convex outer adversarial polytopecited by 567
2017Adversarial Patchcited by 517
2018Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Surveycited by 460

33 more

2017Robust Adversarial Reinforcement Learningcited by 451
2018Efficient Neural Network Robustness Certification with General Activation Functionscited by 445
2018On the Effectiveness of Interval Bound Propagation for Training Verifiably Robust Modelscited by 328
2017Adversarial Attacks on Neural Network Policiescited by 320
2018Adversarial Attacks and Defences: A Surveycited by 259
2020Robust Deep Reinforcement Learning against Adversarial Perturbations on State Observationscited by 238
2019Adversarial Policies: Attacking Deep Reinforcement Learningcited by 214
2019Towards Stable and Efficient Training of Verifiably Robust Neural Networkscited by 205
2019Action Robust Reinforcement Learning and Applications in Continuous Controlcited by 174
2017Robust Deep Reinforcement Learning with Adversarial Attackscited by 154
2021Robust Reinforcement Learning on State Observations with Learned Optimal Adversarycited by 135
2017Delving into adversarial attacks on deep policiescited by 104
2017Vulnerability of Deep Reinforcement Learning to Policy Induction Attackscited by 91
2020Adaptive Reward-Poisoning Attacks against Reinforcement Learningcited by 86
2020Policy Teaching via Environment Poisoning: Training-time Adversarial Attacks against Reinforcement Learningcited by 81
2019Robust Reinforcement Learning for Continuous Control with Model Misspecificationcited by 78
2020Robust Deep Reinforcement Learning through Adversarial Losscited by 75
2022Safe Reinforcement Learning by Imagining the Near Futurecited by 67
2021Who Is the Strongest Enemy? Towards Optimal and Efficient Evasion Attacks in Deep RLcited by 55
2019Deceptive Reinforcement Learning Under Adversarial Manipulations on Cost Signalscited by 50
2019Certified Adversarial Robustness for Deep Reinforcement Learningcited by 47
2021CROP: Certifying Robust Policies for Reinforcement Learning through Functional Smoothingcited by 42
2021Policy Smoothing for Provably Robust Reinforcement Learningcited by 40
2020Vulnerability-Aware Poisoning Mechanism for Online RL with Unknown Dynamicscited by 38
2017Whatever Does Not Kill Deep Reinforcement Learning, Makes It Strongercited by 35
2019Characterizing Attacks on Deep Reinforcement Learningcited by 31
2020Robustifying Reinforcement Learning Agents via Action Space Adversarial Trainingcited by 30
2019The Value Function Polytope in Reinforcement Learningcited by 29
2019Online Robustness Training for Deep Reinforcement Learningcited by 26
2013Scaling Up Robust MDPs by Reinforcement Learningcited by 17
2020Query-based Targeted Action-Space Adversarial Policies on Deep Reinforcement Learning Agentscited by 15
2021Investigating Vulnerabilities of Deep Neural Policiescited by 15
2019Curious iLQR: Resolving Uncertainty in Model-based RLcited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 1; 2023: 10; 2024: 14; 2025: 7; 2026: 11.

43 papers.

Sort:

By area:

2022What is the Solution for State-Adversarial Multi-Agent Reinforcement Learning?cited by 22
2024RAT: Adversarial Attacks on Deep Reinforcement Agents for Targeted Behaviorscited by 16
2026Failure-Aware RL: Reliable Offline-to-Online Reinforcement Learning with Self-Recovery for Real-World Manipulationcited by 11
2023Adjustable Robust Reinforcement Learning for Online 3D Bin Packingcited by 10
2024Beyond Worst-case Attacks: Robust RL with Adaptive Defense via Non-dominated Policiescited by 8
2023Rethinking Adversarial Policies: A Generalized Attack Formulation and Provable Defense in RLcited by 7
2023Improve Robustness of Reinforcement Learning against Observation Perturbations via l_∞ Lipschitz Policy Networkscited by 7
2024Mitigating Adversarial Perturbations for Deep Reinforcement Learning via Vector Quantizationcited by 6
2024Adversarial Training: A Surveycited by 6
2024Towards Optimal Adversarial Robust Q-learning with Bellman Infinity-errorcited by 4
2024Breaking the Barrier: Enhanced Utility and Robustness in Smoothed DRL Agentscited by 4
2023Regret-Based Defense in Adversarial Reinforcement Learningcited by 3

31 more

2023Toward Evaluating Robustness of Reinforcement Learning with Adversarial Policycited by 3
2023Robust Multi-agent Communication via Multi-view Message Certificationcited by 3
2024Belief-Enriched Pessimistic Q-Learning against Adversarial State Perturbationscited by 2
2024Specification Overfitting in Artificial Intelligencecited by 2
2024Robust Deep Reinforcement Learning against Adversarial Behavior Manipulationcited by 2
2024On Minimizing Adversarial Counterfactual Error in Adversarial RLcited by 2
2026Stealthy World Model Manipulation via Data Poisoningcited by 2
2023Robustness to Multi-Modal Environment Uncertainty in MARL using Curriculum Learningcited by 1
2025Action Robust Reinforcement Learning via Optimal Adversary Aware Policy Optimizationcited by 1
2026Safe and Robust Neural Policy Learning with Statistical Verification for Sim-to-Real Deployment in Roboticscited by 1
2023Game-Theoretic Robust Reinforcement Learning Handles Temporally-Coupled Perturbationscited by 0
2023Robust Offline Reinforcement Learning -- Certify the Confidence Intervalcited by 0
2023Safe Reinforcement Learning in Tensor Reproducing Kernel Hilbert Spacecited by 0
2024On the Perturbed States for Transformed Input-robust Reinforcement Learningcited by 0
2024Robust off-policy Reinforcement Learning via Soft Constrained Adversarycited by 0
2024Constrained Optimal Fuel Consumption of HEVs under Observational Noisecited by 0
2024Optimizing Sensor Redundancy in Sequential Decision-Making Problemscited by 0
2025Robust and Safe Multi-Agent Reinforcement Learning with Communication for Autonomous Vehicles: From Simulation to Hardwarecited by 0
2025Towards Robust Deep Reinforcement Learning against Environmental State Perturbationcited by 0
2025Off-Policy Actor-Critic for Adversarial Observation Robustness: Virtual Alternative Training via Symmetric Policy Evaluationcited by 0
2025Reducing the Probability of Undesirable Outputs in Language Models Using Probabilistic Inferencecited by 0
2025Diffusion Guided Adversarial State Perturbations in Reinforcement Learningcited by 0
2025On the Tension Between Optimality and Adversarial Robustness in Policy Optimizationcited by 0
2026Fuz-RL: A Fuzzy-Guided Robust Framework for Safe Reinforcement Learning under Uncertaintycited by 0
2026Entropy Regularization Improves Policy Robustness in Continuous-Time Reinforcement Learningcited by 0
2026Robust Critics: Defending LLMs Against Multi-Turn Attackscited by 0
2026CIVA: Critic-Induced Value-Subspace Attacks on Visual World-Model Agentscited by 0
2026Certifying Lower Bounds for Risk-Sensitive Reinforcement Learning under Adversarial State Perturbationscited by 0
2026Robust Policy Optimization via Adversarial Importance Samplingcited by 0
2026A Unified and Constrained View of Regularization-Based Robust Reinforcement Learningcited by 0
2026FoSeRL: Formal Sequential Robustness Certification for Reinforcement Learning Policiescited by 0