# Stealing Deep Reinforcement Learning Models for Fun and Profit

arXiv: https://arxiv.org/abs/2006.05032 · HTML: https://mlsanity.com/paper/2006.05032/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Placed in Offline RL algorithms, one of 13 directions in Offline RL and imitation learning from data.
- First posted on arXiv 9 Jun 2020.

## Placed in

- Offline RL and imitation learning from data / [Offline RL algorithms](https://mlsanity.com/offlinerl/offline-rl-algorithms/)

## Built on by (most cited first, 24 of 24)

- [I Know What You Trained Last Summer: A Survey on Stealing Machine Learning Models and Defences](https://mlsanity.com/paper/2206.08451/) (2022)
- [Towards more Practical Threat Models in Artificial Intelligence Security](https://mlsanity.com/paper/2311.09994/) (2023)
- [A Search-Based Testing Approach for Deep Reinforcement Learning Agents](https://mlsanity.com/paper/2206.07813/) (2022)
- [ORL-AUDITOR: Dataset Auditing in Offline Deep Reinforcement Learning](https://mlsanity.com/paper/2309.03081/) (2023)
- [Model Inversion Attacks: A Survey of Approaches and Countermeasures](https://mlsanity.com/paper/2411.10023/) (2024)
- [SMARLA: A Safety Monitoring Approach for Deep Reinforcement Learning Agents](https://mlsanity.com/paper/2308.02594/) (2023)
- [Model Extraction and Defenses on Generative Adversarial Networks](https://mlsanity.com/paper/2101.02069/) (2021)
- [Confidential Machine Learning Computation in Untrusted Environments: A Systems Security Perspective](https://mlsanity.com/paper/2111.03308/) (2021)
- [The Security of Deep Learning Defences for Medical Imaging](https://mlsanity.com/paper/2201.08661/) (2022)
- [Privacy of Autonomous Vehicles: Risks, Protection Methods, and Future Directions](https://mlsanity.com/paper/2209.04022/) (2022)
- [Mercury: An Automated Remote Side-channel Attack to Nvidia Deep Learning Accelerator](https://mlsanity.com/paper/2308.01193/) (2023)
- [SAME: Sample Reconstruction against Model Extraction Attacks](https://mlsanity.com/paper/2312.10578/) (2023)
- [HODA: Hardness-Oriented Detection of Model Extraction Attacks](https://mlsanity.com/paper/2106.11424/) (2021)
- [Stealthy Imitation: Reward-guided Environment-free Policy Stealing](https://mlsanity.com/paper/2405.07004/) (2024)
- [Security of Deep Reinforcement Learning for Autonomous Driving: A Survey](https://mlsanity.com/paper/2212.06123/) (2022)
- [FLARE: Fingerprinting Deep Reinforcement Learning Agents using Universal Adversarial Masks](https://mlsanity.com/paper/2307.14751/) (2023)
- [Belief-Enriched Pessimistic Q-Learning against Adversarial State Perturbations](https://mlsanity.com/paper/2403.04050/) (2024)
- [Enhancing Security in Deep Reinforcement Learning: A Comprehensive Survey on Adversarial Attacks and Defenses](https://mlsanity.com/paper/2510.20314/) (2025)
- [First to Possess His Statistics: Data-Free Model Extraction Attack on Tabular Data](https://mlsanity.com/paper/2109.14857/) (2021)
- [Angel or Demon: Investigating the Plasticity Interventions' Impact on Backdoor Threats in Deep Reinforcement Learning](https://mlsanity.com/paper/2605.14587/) (2026)
- [Mitigation of Adversarial Policy Imitation via Constrained Randomization of Policy (CRoP)](https://mlsanity.com/paper/2109.14678/) (2021)
- [Enhancing Digital Health Services: A Machine Learning Approach to Personalized Exercise Goal Setting](https://mlsanity.com/paper/2204.00961/) (2022)
- [Attackers Can Do Better: Over- and Understated Factors of Model Stealing Attacks](https://mlsanity.com/paper/2503.06188/) (2025)
- [Diffusion Guided Adversarial State Perturbations in Reinforcement Learning](https://mlsanity.com/paper/2511.07701/) (2025)

## Abstract (arXiv)

This paper presents the first model extraction attack against Deep Reinforcement Learning (DRL), which enables an external adversary to precisely recover a black-box DRL model only from its interaction with the environment. Model extraction attacks against supervised Deep Learning models have been widely studied. However, those techniques cannot be applied to the reinforcement learning scenario due to DRL models' high complexity, stochasticity and limited observable information. We propose a novel methodology to overcome the above challenges. The key insight of our approach is that the process of DRL model extraction is equivalent to imitation learning, a well-established solution to learn sequential decision-making policies. Based on this observation, our methodology first builds a classifier to reveal the training algorithm family of the targeted black-box DRL model only based on its predicted actions, and then leverages state-of-the-art imitation learning techniques to replicate the model from the identified algorithm family. Experimental results indicate that our methodology can effectively recover the DRL models with high fidelity and accuracy. We also demonstrate two use cases to show that our model extraction attack can (1) significantly improve the success rate of adversarial attacks, and (2) steal DRL models stealthily even they are protected by DNN watermarks. These pose a severe threat to the intellectual property and privacy protection of DRL applications.