# Effectiveness of Distillation Attack and Countermeasure on Neural Network Watermarking

arXiv: https://arxiv.org/abs/1906.06046 · HTML: https://mlsanity.com/paper/1906.06046/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Jun 2019.

## Built on by (most cited first, 12 of 12)

- [Dataset Inference: Ownership Resolution in Machine Learning](https://mlsanity.com/paper/2104.10706/) (2021)
- [Rethinking White-Box Watermarks on Deep Learning Models under Neural Structural Obfuscation](https://mlsanity.com/paper/2303.09732/) (2023)
- [Deep Intellectual Property Protection: A Survey](https://mlsanity.com/paper/2304.14613/) (2023)
- [Identifying Appropriate Intellectual Property Protection Mechanisms for Machine Learning Models: A Systematization of Watermarking, Fingerprinting, Model Access, and Attacks](https://mlsanity.com/paper/2304.11285/) (2023)
- [Detect and remove watermark in deep neural networks via generative adversarial networks](https://mlsanity.com/paper/2106.08104/) (2021)
- [HufuNet: Embedding the Left Piece as Watermark and Keeping the Right Piece for Ownership Verification in Deep Neural Networks](https://mlsanity.com/paper/2103.13628/) (2021)
- [TATTOOED: A Robust Deep Neural Network Watermarking Scheme based on Spread-Spectrum Channel Coding](https://mlsanity.com/paper/2202.06091/) (2022)
- [Revisiting Black-Box Model Ownership Verification through Information Theory](https://mlsanity.com/paper/2409.06130/) (2024)
- [Neural Dehydration: Effective Erasure of Black-box Watermarks from DNNs with Limited Data](https://mlsanity.com/paper/2309.03466/) (2023)
- [A Secure Fingerprinting Framework for Distributed Image Classification](https://mlsanity.com/paper/2207.04668/) (2022)
- [VeriDIP: Verifying Ownership of Deep Neural Networks through Privacy Leakage Fingerprints](https://mlsanity.com/paper/2310.10656/) (2023)
- [Watermarking Recommender Systems](https://mlsanity.com/paper/2407.21034/) (2024)

## Abstract (arXiv)

The rise of machine learning as a service and model sharing platforms has raised the need of traitor-tracing the models and proof of authorship. Watermarking technique is the main component of existing methods for protecting copyright of models. In this paper, we show that distillation, a widely used transformation technique, is a quite effective attack to remove watermark embedded by existing algorithms. The fragility is due to the fact that distillation does not retain the watermark embedded in the model that is redundant and independent to the main learning task. We design ingrain in response to the destructive distillation. It regularizes a neural network with an ingrainer model, which contains the watermark, and forces the model to also represent the knowledge of the ingrainer. Our extensive evaluations show that ingrain is more robust to distillation attack and its robustness against other widely used transformation techniques is comparable to existing methods.