# Preventing Distillation-based Attacks on Neural Network IP

arXiv: https://arxiv.org/abs/2204.00292 · HTML: https://mlsanity.com/paper/2204.00292/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 8 of its 15 references that are in the map.
- First posted on arXiv 1 Apr 2022.

## Builds on (most cited first, 8 of 8)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Knockoff Nets: Stealing Functionality of Black-Box Models](https://mlsanity.com/paper/1812.02766/) (2018)
- [Fast inference of deep neural networks in FPGAs for particle physics](https://mlsanity.com/paper/1804.06913/) (2018)
- [Prediction Poisoning: Towards Defenses Against DNN Model Stealing Attacks](https://mlsanity.com/paper/1906.10908/) (2019)
- [Hardware and Software Optimizations for Accelerating Deep Neural Networks: Survey of Current Trends, Challenges, and the Road Ahead](https://mlsanity.com/paper/2012.11233/) (2020)
- [Defending Against Machine Learning Model Stealing Attacks Using Deceptive Perturbations](https://mlsanity.com/paper/1806.00054/) (2018)
- [DeepObfuscation: Securing the Structure of Convolutional Neural Networks via Knowledge Distillation](https://mlsanity.com/paper/1806.10313/) (2018)
- [Modeling Techniques for Logic Locking](https://mlsanity.com/paper/2009.10131/) (2020)

## Abstract (arXiv)

Neural networks (NNs) are already deployed in hardware today, becoming valuable intellectual property (IP) as many hours are invested in their training and optimization. Therefore, attackers may be interested in copying, reverse engineering, or even modifying this IP. The current practices in hardware obfuscation, including the widely studied logic locking technique, are insufficient to protect the actual IP of a well-trained NN: its weights. Simply hiding the weights behind a key-based scheme is inefficient (resource-hungry) and inadequate (attackers can exploit knowledge distillation). This paper proposes an intuitive method to poison the predictions that prevent distillation-based attacks; this is the first work to consider such a poisoning approach in hardware-implemented NNs. The proposed technique obfuscates a NN so an attacker cannot train the NN entirely or accurately. We elaborate a threat model which highlights the difference between random logic obfuscation and the obfuscation of NN IP. Based on this threat model, our security analysis shows that the poisoning successfully and significantly reduces the accuracy of the stolen NN model on various representative datasets. Moreover, the accuracy and prediction distributions are maintained, no functionality is disturbed, nor are high overheads incurred. Finally, we highlight that our proposed approach is flexible and does not require manipulation of the NN toolchain.