# Whatever Does Not Kill Deep Reinforcement Learning, Makes It Stronger

arXiv: https://arxiv.org/abs/1712.09344 · HTML: https://mlsanity.com/paper/1712.09344/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Dec 2017.

## Built on by (most cited first, 30 of 35)

- [Robust Reinforcement Learning on State Observations with Learned Optimal Adversary](https://mlsanity.com/paper/2101.08452/) (2021)
- [Efficient Adversarial Training without Attacking: Worst-Case-Aware Robust Reinforcement Learning](https://mlsanity.com/paper/2210.05927/) (2022)
- [CROP: Certifying Robust Policies for Reinforcement Learning through Functional Smoothing](https://mlsanity.com/paper/2106.09292/) (2021)
- [Trustworthy Reinforcement Learning Against Intrinsic Vulnerabilities: Robustness, Safety, and Generalizability](https://mlsanity.com/paper/2209.08025/) (2022)
- [COPA: Certifying Robust Policies for Offline Reinforcement Learning against Poisoning Attacks](https://mlsanity.com/paper/2203.08398/) (2022)
- [Defense Against Reward Poisoning Attacks in Reinforcement Learning](https://mlsanity.com/paper/2102.05776/) (2021)
- [Robust Deep Reinforcement Learning through Bootstrapped Opportunistic Curriculum](https://mlsanity.com/paper/2206.10057/) (2022)
- [Implicit Poisoning Attacks in Two-Agent Reinforcement Learning: Adversarial Policies for Training-Time Attacks](https://mlsanity.com/paper/2302.13851/) (2023)
- [A Search-Based Testing Approach for Deep Reinforcement Learning Agents](https://mlsanity.com/paper/2206.07813/) (2022)
- [Resilient Machine Learning for Networked Cyber Physical Systems: A Survey for Machine Learning Security to Securing Machine Learning for CPS](https://mlsanity.com/paper/2102.07244/) (2021)
- [Adjustable Robust Reinforcement Learning for Online 3D Bin Packing](https://mlsanity.com/paper/2310.04323/) (2023)
- [Real-time Adversarial Perturbations against Deep Reinforcement Learning Policies: Attacks and Defenses](https://mlsanity.com/paper/2106.08746/) (2021)
- [Efficient Reward Poisoning Attacks on Online Deep Reinforcement Learning](https://mlsanity.com/paper/2205.14842/) (2022)
- [Mitigating Adversarial Perturbations for Deep Reinforcement Learning via Vector Quantization](https://mlsanity.com/paper/2410.03376/) (2024)
- [Deep-Attack over the Deep Reinforcement Learning](https://mlsanity.com/paper/2205.00807/) (2022)
- [Black-Box Targeted Reward Poisoning Attack Against Online Deep Reinforcement Learning](https://mlsanity.com/paper/2305.10681/) (2023)
- [Towards Optimal Adversarial Robust Q-learning with Bellman Infinity-error](https://mlsanity.com/paper/2402.02165/) (2024)
- [SoK: Adversarial Machine Learning Attacks and Defences in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2301.04299/) (2023)
- [Toward Evaluating Robustness of Reinforcement Learning with Adversarial Policy](https://mlsanity.com/paper/2305.02605/) (2023)
- [Advancing Robustness in Deep Reinforcement Learning with an Ensemble Defense Approach](https://mlsanity.com/paper/2507.17070/) (2025)
- [On Assessing The Safety of Reinforcement Learning algorithms Using Formal Methods](https://mlsanity.com/paper/2111.04865/) (2021)
- [Policy Teaching via Data Poisoning in Learning from Human Preferences](https://mlsanity.com/paper/2503.10228/) (2025)
- [A Conceptual Framework for AI-based Decision Systems in Critical Infrastructures](https://mlsanity.com/paper/2504.16133/) (2025)
- [Policy Resilience to Environment Poisoning Attacks on Reinforcement Learning](https://mlsanity.com/paper/2304.12151/) (2023)
- [Attacking and Defending Deep Reinforcement Learning Policies](https://mlsanity.com/paper/2205.07626/) (2022)
- [Explainable and Safe Reinforcement Learning for Autonomous Air Mobility](https://mlsanity.com/paper/2211.13474/) (2022)
- [Game-Theoretic Robust Reinforcement Learning Handles Temporally-Coupled Perturbations](https://mlsanity.com/paper/2307.12062/) (2023)
- [On the Perturbed States for Transformed Input-robust Reinforcement Learning](https://mlsanity.com/paper/2408.00023/) (2024)
- [Robust off-policy Reinforcement Learning via Soft Constrained Adversary](https://mlsanity.com/paper/2409.00418/) (2024)
- [On the Definition of Robustness and Resilience of AI Agents for Real-time Congestion Management](https://mlsanity.com/paper/2504.13314/) (2025)

## Abstract (arXiv)

Recent developments have established the vulnerability of deep Reinforcement Learning (RL) to policy manipulation attacks via adversarial perturbations. In this paper, we investigate the robustness and resilience of deep RL to training-time and test-time attacks. Through experimental results, we demonstrate that under noncontiguous training-time attacks, Deep Q-Network (DQN) agents can recover and adapt to the adversarial conditions by reactively adjusting the policy. Our results also show that policies learned under adversarial perturbations are more robust to test-time attacks. Furthermore, we compare the performance of ε-greedy and parameter-space noise exploration methods in terms of robustness and resilience against adversarial perturbations.