# Mitigation of Adversarial Policy Imitation via Constrained Randomization of Policy (CRoP)

arXiv: https://arxiv.org/abs/2109.14678 · HTML: https://mlsanity.com/paper/2109.14678/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 8 of its 15 references that are in the map.
- First posted on arXiv 29 Sep 2021.

## Placed in

- Offline RL and imitation learning from data / [Imitation learning methods](https://mlsanity.com/offlinerl/imitation-learning-methods/)

## Builds on (most cited first, 8 of 8)

- [Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems](https://mlsanity.com/paper/2005.01643/) (2020)
- [Generative Adversarial Imitation Learning](https://mlsanity.com/paper/1606.03476/) (2016)
- [Imitation Learning as f-Divergence Minimization](https://mlsanity.com/paper/1905.12888/) (2019)
- [Stealing Deep Reinforcement Learning Models for Fun and Profit](https://mlsanity.com/paper/2006.05032/) (2020)
- [Adversarial Exploitation of Policy Imitation](https://mlsanity.com/paper/1906.01121/) (2019)
- [Sequential Triggers for Watermarking of Deep Reinforcement Learning Policies](https://mlsanity.com/paper/1906.01126/) (2019)
- [The Faults in Our Pi Stars: Security Issues and Open Challenges in Deep Reinforcement Learning](https://mlsanity.com/paper/1810.10369/) (2018)
- [PAC Bounds for Imitation and Model-based Batch Learning of Contextual Markov Decision Processes](https://mlsanity.com/paper/2006.06352/) (2020)

## Abstract (arXiv)

Deep reinforcement learning (DRL) policies are vulnerable to unauthorized replication attacks, where an adversary exploits imitation learning to reproduce target policies from observed behavior. In this paper, we propose Constrained Randomization of Policy (CRoP) as a mitigation technique against such attacks. CRoP induces the execution of sub-optimal actions at random under performance loss constraints. We present a parametric analysis of CRoP, address the optimality of CRoP, and establish theoretical bounds on the adversarial budget and the expectation of loss. Furthermore, we report the experimental evaluation of CRoP in Atari environments under adversarial imitation, which demonstrate the efficacy and feasibility of our proposed method against policy replication attacks.