# Minimalistic Attacks: How Little it Takes to Fool a Deep Reinforcement Learning Policy

arXiv: https://arxiv.org/abs/1911.03849 · HTML: https://mlsanity.com/paper/1911.03849/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Nov 2019.

## Built on by (most cited first, 10 of 10)

- [RAPid-Learn: A Framework for Learning to Recover for Handling Novelties in Open-World Environments](https://mlsanity.com/paper/2206.12493/) (2022)
- [Resilient Machine Learning for Networked Cyber Physical Systems: A Survey for Machine Learning Security to Securing Machine Learning for CPS](https://mlsanity.com/paper/2102.07244/) (2021)
- [Machine Learning for Security in Vehicular Networks: A Comprehensive Survey](https://mlsanity.com/paper/2105.15035/) (2021)
- [A Framework for Few-Shot Policy Transfer through Observation Mapping and Behavior Cloning](https://mlsanity.com/paper/2310.08836/) (2023)
- [Towards Resilient Artificial Intelligence: Survey and Research Issues](https://mlsanity.com/paper/2109.08904/) (2021)
- [Brick Tic-Tac-Toe: Exploring the Generalizability of AlphaZero to Novel Test Environments](https://mlsanity.com/paper/2207.05991/) (2022)
- [Quality Diversity for Robot Learning: Limitations and Future Directions](https://mlsanity.com/paper/2407.17515/) (2024)
- [Knowledge Transfer in Deep Reinforcement Learning via an RL-Specific GAN-Based Correspondence Function](https://mlsanity.com/paper/2209.06604/) (2022)
- [Collapsing Sequence-Level Data-Policy Coverage via Poisoning Attack in Offline Reinforcement Learning](https://mlsanity.com/paper/2506.11172/) (2025)
- [Breaking Task Impasses Quickly: Adaptive Neuro-Symbolic Learning for Open-World Robotics](https://mlsanity.com/paper/2601.16985/) (2026)

## Abstract (arXiv)

Recent studies have revealed that neural network-based policies can be easily fooled by adversarial examples. However, while most prior works analyze the effects of perturbing every pixel of every frame assuming white-box policy access, in this paper we take a more restrictive view towards adversary generation - with the goal of unveiling the limits of a model's vulnerability. In particular, we explore minimalistic attacks by defining three key settings: (1) black-box policy access: where the attacker only has access to the input (state) and output (action probability) of an RL policy; (2) fractional-state adversary: where only several pixels are perturbed, with the extreme case being a single-pixel adversary; and (3) tactically-chanced attack: where only significant frames are tactically chosen to be attacked. We formulate the adversarial attack by accommodating the three key settings and explore their potency on six Atari games by examining four fully trained state-of-the-art policies. In Breakout, for example, we surprisingly find that: (i) all policies showcase significant performance degradation by merely modifying 0.01% of the input state, and (ii) the policy trained by DQN is totally deceived by perturbation to only 1% frames.