# Blackbox Attacks on Reinforcement Learning Agents Using Approximated Temporal Information

arXiv: https://arxiv.org/abs/1909.02918 · HTML: https://mlsanity.com/paper/1909.02918/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Sep 2019.

## Built on by (most cited first, 13 of 13)

- [Generalization of Reinforcement Learning with Policy-Aware Adversarial Data Augmentation](https://mlsanity.com/paper/2106.15587/) (2021)
- [Resilient Machine Learning for Networked Cyber Physical Systems: A Survey for Machine Learning Security to Securing Machine Learning for CPS](https://mlsanity.com/paper/2102.07244/) (2021)
- [Real-time Adversarial Perturbations against Deep Reinforcement Learning Policies: Attacks and Defenses](https://mlsanity.com/paper/2106.08746/) (2021)
- [Efficient Adversarial Training With Data Pruning](https://mlsanity.com/paper/2207.00694/) (2022)
- [New Challenges in Reinforcement Learning: A Survey of Security and Privacy](https://mlsanity.com/paper/2301.00188/) (2022)
- [Cooperative Backdoor Attack in Decentralized Reinforcement Learning with Theoretical Guarantee](https://mlsanity.com/paper/2405.15245/) (2024)
- [Deep-Attack over the Deep Reinforcement Learning](https://mlsanity.com/paper/2205.00807/) (2022)
- [A Survey of Machine Learning-Based Ride-Hailing Planning](https://mlsanity.com/paper/2303.14646/) (2023)
- [Imitating Opponent to Win: Adversarial Policy Imitation Learning in Two-player Competitive Games](https://mlsanity.com/paper/2210.16915/) (2022)
- [Security of Deep Reinforcement Learning for Autonomous Driving: A Survey](https://mlsanity.com/paper/2212.06123/) (2022)
- [Learned-Database Systems Security](https://mlsanity.com/paper/2212.10318/) (2022)
- [Adversarial Reinforcement Learning in Dynamic Channel Access and Power Control](https://mlsanity.com/paper/2105.05817/) (2021)
- [Adversarial Attacks on Reinforcement Learning-based Medical Questionnaire Systems: Input-level Perturbation Strategies and Medical Constraint Validation](https://mlsanity.com/paper/2508.05677/) (2025)

## Abstract (arXiv)

Recent research on reinforcement learning (RL) has suggested that trained agents are vulnerable to maliciously crafted adversarial samples. In this work, we show how such samples can be generalised from White-box and Grey-box attacks to a strong Black-box case, where the attacker has no knowledge of the agents, their training parameters and their training methods. We use sequence-to-sequence models to predict a single action or a sequence of future actions that a trained agent will make. First, we show our approximation model, based on time-series information from the agent, consistently predicts RL agents' future actions with high accuracy in a Black-box setup on a wide range of games and RL algorithms. Second, we find that although adversarial samples are transferable from the target model to our RL agents, they often outperform random Gaussian noise only marginally. This highlights a serious methodological deficiency in previous work on such agents; random jamming should have been taken as the baseline for evaluation. Third, we propose a novel use for adversarial samplesin Black-box attacks of RL agents: they can be used to trigger a trained agent to misbehave after a specific time delay. This appears to be a genuinely new type of attack. It potentially enables an attacker to use devices controlled by RL agents as time bombs.