# FLARE: Fingerprinting Deep Reinforcement Learning Agents using Universal Adversarial Masks

arXiv: https://arxiv.org/abs/2307.14751 · HTML: https://mlsanity.com/paper/2307.14751/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 50 references that are in the map.
- First posted on arXiv 27 Jul 2023.

## Builds on (most cited first, 30 of 36)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Asynchronous Methods for Deep Reinforcement Learning](https://mlsanity.com/paper/1602.01783/) (2016)
- [Learning both Weights and Connections for Efficient Neural Networks](https://mlsanity.com/paper/1506.02626/) (2015)
- [Dueling Network Architectures for Deep Reinforcement Learning](https://mlsanity.com/paper/1511.06581/) (2015)
- [Universal adversarial perturbations](https://mlsanity.com/paper/1610.08401/) (2016)
- [QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation](https://mlsanity.com/paper/1806.10293/) (2018)
- [Universal Adversarial Triggers for Attacking and Analyzing NLP](https://mlsanity.com/paper/1908.07125/) (2019)
- [Deep Reinforcement Learning for Autonomous Driving: A Survey](https://mlsanity.com/paper/2002.00444/) (2020)
- [Turning Your Weakness Into a Strength: Watermarking Deep Neural Networks by Backdooring](https://mlsanity.com/paper/1802.04633/) (2018)
- [Adversarial Attacks on Neural Network Policies](https://mlsanity.com/paper/1702.02284/) (2017)
- [CNN Features off-the-shelf: an Astounding Baseline for Recognition](https://mlsanity.com/paper/1403.6382/) (2014)
- [Robust Deep Reinforcement Learning against Adversarial Perturbations on State Observations](https://mlsanity.com/paper/2003.08938/) (2020)
- [Adversarial Policies: Attacking Deep Reinforcement Learning](https://mlsanity.com/paper/1905.10615/) (2019)
- [The Space of Transferable Adversarial Examples](https://mlsanity.com/paper/1704.03453/) (2017)
- [DAWN: Dynamic Adversarial Watermarking of Neural Networks](https://mlsanity.com/paper/1906.00830/) (2019)
- [Deep Neural Network Fingerprinting by Conferrable Adversarial Examples](https://mlsanity.com/paper/1912.00888/) (2019)
- [Robust Reinforcement Learning for Continuous Control with Model Misspecification](https://mlsanity.com/paper/1906.07516/) (2019)
- [Robust Deep Reinforcement Learning through Adversarial Loss](https://mlsanity.com/paper/2008.01976/) (2020)
- [IPGuard: Protecting Intellectual Property of Deep Neural Networks via Fingerprinting the Classification Boundary](https://mlsanity.com/paper/1910.12903/) (2019)
- [SoK: How Robust is Image Classification Deep Neural Network Watermarking? (Extended Version)](https://mlsanity.com/paper/2108.04974/) (2021)
- [Graying the black box: Understanding DQNs](https://mlsanity.com/paper/1602.02658/) (2016)
- [Fingerprinting Deep Neural Networks Globally via Universal Adversarial Perturbations](https://mlsanity.com/paper/2202.08602/) (2022)
- [Cache Telepathy: Leveraging Shared Resource Attacks to Learn DNN Architectures](https://mlsanity.com/paper/1808.04761/) (2018)
- [NAG: Network for Adversary Generation](https://mlsanity.com/paper/1712.03390/) (2017)
- [Are Vision Transformers Robust to Patch Perturbations?](https://mlsanity.com/paper/2111.10659/) (2021)
- [Characterizing Attacks on Deep Reinforcement Learning](https://mlsanity.com/paper/1907.09470/) (2019)
- [Stealing Deep Reinforcement Learning Models for Fun and Profit](https://mlsanity.com/paper/2006.05032/) (2020)
- [Towards Better Interpretability in Deep Q-Networks](https://mlsanity.com/paper/1809.05630/) (2018)
- [Transforming Cooling Optimization for Green Data Center via Deep Reinforcement Learning](https://mlsanity.com/paper/1709.05077/) (2017)
- [Detecting Adversarial Attacks on Neural Network Policies with Visual Foresight](https://mlsanity.com/paper/1710.00814/) (2017)

## Built on by (most cited first, 2 of 2)

- [Copyright Protection for Large Language Models: A Survey of Methods, Challenges, and Trends](https://mlsanity.com/paper/2508.11548/) (2025)
- [FedPoP: Federated Learning Meets Proof of Participation](https://mlsanity.com/paper/2511.08207/) (2025)

## Abstract (arXiv)

We propose FLARE, the first fingerprinting mechanism to verify whether a suspected Deep Reinforcement Learning (DRL) policy is an illegitimate copy of another (victim) policy. We first show that it is possible to find non-transferable, universal adversarial masks, i.e., perturbations, to generate adversarial examples that can successfully transfer from a victim policy to its modified versions but not to independently trained policies. FLARE employs these masks as fingerprints to verify the true ownership of stolen DRL policies by measuring an action agreement value over states perturbed by such masks. Our empirical evaluations show that FLARE is effective (100% action agreement on stolen copies) and does not falsely accuse independent policies (no false positives). FLARE is also robust to model modification attacks and cannot be easily evaded by more informed adversaries without negatively impacting agent performance. We also show that not all universal adversarial masks are suitable candidates for fingerprints due to the inherent characteristics of DRL policies. The spatio-temporal dynamics of DRL problems and sequential decision-making process make characterizing the decision boundary of DRL policies more difficult, as well as searching for universal masks that capture the geometry of it.