# Copy, Right? A Testing Framework for Copyright Protection of Deep Learning Models

arXiv: https://arxiv.org/abs/2112.05588 · HTML: https://mlsanity.com/paper/2112.05588/

## Key facts

- Cited by 46 papers in the map (data through 14 Oct 2026).
- Placed in Watermarking generated content and models, one of 12 directions in LLM safety and security.
- Builds on 34 of its 48 references that are in the map.
- First posted on arXiv 10 Dec 2021.

## Placed in

- LLM safety and security / [Watermarking generated content and models](https://mlsanity.com/safety/watermarking-generated-content-and-models/)

## Builds on (most cited first, 30 of 34)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Towards Evaluating the Robustness of Neural Networks](https://mlsanity.com/paper/1608.04644/) (2016)
- [A Survey on Bias and Fairness in Machine Learning](https://mlsanity.com/paper/1908.09635/) (2019)
- [Theoretically Principled Trade-off between Robustness and Accuracy](https://mlsanity.com/paper/1901.08573/) (2019)
- [Practical Black-Box Attacks against Machine Learning](https://mlsanity.com/paper/1602.02697/) (2016)
- [Robustness May Be at Odds with Accuracy](https://mlsanity.com/paper/1805.12152/) (2018)
- [Speech Commands: A Dataset for Limited-Vocabulary Speech Recognition](https://mlsanity.com/paper/1804.03209/) (2018)
- [Speech Recognition with Deep Recurrent Neural Networks](https://mlsanity.com/paper/1303.5778/) (2013)
- [Stealing Machine Learning Models via Prediction APIs](https://mlsanity.com/paper/1609.02943/) (2016)
- [Rethinking the Value of Network Pruning](https://mlsanity.com/paper/1810.05270/) (2018)
- [Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networks](https://mlsanity.com/paper/1805.12185/) (2018)
- [Turning Your Weakness Into a Strength: Watermarking Deep Neural Networks by Backdooring](https://mlsanity.com/paper/1802.04633/) (2018)
- [Understanding Neural Networks Through Deep Visualization](https://mlsanity.com/paper/1506.06579/) (2015)
- [DeepXplore: Automated Whitebox Testing of Deep Learning Systems](https://mlsanity.com/paper/1705.06640/) (2017)
- [Knockoff Nets: Stealing Functionality of Black-Box Models](https://mlsanity.com/paper/1812.02766/) (2018)
- [Embedding Watermarks into Deep Neural Networks](https://mlsanity.com/paper/1701.04082/) (2017)
- [Comparing Rewinding and Fine-tuning in Neural Network Pruning](https://mlsanity.com/paper/2003.02389/) (2020)
- [DeepDriving: Learning Affordance for Direct Perception in Autonomous Driving](https://mlsanity.com/paper/1505.00256/) (2015)
- [High Accuracy and High Fidelity Extraction of Neural Networks](https://mlsanity.com/paper/1909.01838/) (2019)
- [Skip Connections Matter: On the Transferability of Adversarial Examples Generated with ResNets](https://mlsanity.com/paper/2002.05990/) (2020)
- [PRADA: Protecting against DNN Model Stealing Attacks](https://mlsanity.com/paper/1805.02628/) (2018)
- [Entangled Watermarks as a Defense against Model Extraction](https://mlsanity.com/paper/2002.12200/) (2020)
- [Adversarial Frontier Stitching for Remote Neural Network Watermarking](https://mlsanity.com/paper/1711.01894/) (2017)
- [Dataset Inference: Ownership Resolution in Machine Learning](https://mlsanity.com/paper/2104.10706/) (2021)
- [The Cost of Training NLP Models: A Concise Overview](https://mlsanity.com/paper/2004.08900/) (2020)
- [Deep Neural Network Fingerprinting by Conferrable Adversarial Examples](https://mlsanity.com/paper/1912.00888/) (2019)
- [[Extended version] Rethinking Deep Neural Network Ownership Verification: Embedding Passports to Defeat Ambiguity Attacks](https://mlsanity.com/paper/1909.07830/) (2019)

## Built on by (most cited first, 30 of 46)

- [SSLGuard: A Watermarking Scheme for Self-supervised Learning Pre-trained Encoders](https://mlsanity.com/paper/2201.11692/) (2022)
- [Explanation as a Watermark: Towards Harmless and Multi-bit Model Ownership Verification via Watermarking Feature Attribution](https://mlsanity.com/paper/2405.04825/) (2024)
- [Distillation-Resistant Watermarking for Model Protection in NLP](https://mlsanity.com/paper/2210.03312/) (2022)
- [We Should Chart an Atlas of All the World's Models](https://mlsanity.com/paper/2503.10633/) (2025)
- [Are You Stealing My Model? Sample Correlation for Fingerprinting Deep Neural Networks](https://mlsanity.com/paper/2210.15427/) (2022)
- [No Privacy Left Outside: On the (In-)Security of TEE-Shielded DNN Partition for On-Device ML](https://mlsanity.com/paper/2310.07152/) (2023)
- [Rethinking White-Box Watermarks on Deep Learning Models under Neural Structural Obfuscation](https://mlsanity.com/paper/2303.09732/) (2023)
- [ProFLingo: A Fingerprinting-based Intellectual Property Protection Scheme for Large Language Models](https://mlsanity.com/paper/2405.02466/) (2024)
- [Copyright Protection and Accountability of Generative AI:Attack, Watermarking and Attribution](https://mlsanity.com/paper/2303.09272/) (2023)
- [SoK: Dataset Copyright Auditing in Machine Learning Systems](https://mlsanity.com/paper/2410.16618/) (2024)
- [Model Provenance Testing for Large Language Models](https://mlsanity.com/paper/2502.00706/) (2025)
- [SoK: Large Language Model Copyright Auditing via Fingerprinting](https://mlsanity.com/paper/2508.19843/) (2025)
- [ORL-AUDITOR: Dataset Auditing in Offline Deep Reinforcement Learning](https://mlsanity.com/paper/2309.03081/) (2023)
- [Towards Robust Model Watermark via Reducing Parametric Vulnerability](https://mlsanity.com/paper/2309.04777/) (2023)
- [FIT-Print: Towards False-claim-resistant Model Ownership Verification via Targeted Fingerprint](https://mlsanity.com/paper/2501.15509/) (2025)
- [RAFP: Identifying LLM Lineages via Rare-Region Fingerprints](https://mlsanity.com/paper/2505.12682/) (2025)
- [Are Robust LLM Fingerprints Adversarially Robust?](https://mlsanity.com/paper/2509.26598/) (2025)
- [ModelShield: Adaptive and Robust Watermark against Model Extraction Attack](https://mlsanity.com/paper/2405.02365/) (2024)
- [Cracking White-box DNN Watermarks via Invariant Neuron Transforms](https://mlsanity.com/paper/2205.00199/) (2022)
- [ModelLock: Locking Your Model With a Spell](https://mlsanity.com/paper/2405.16285/) (2024)
- [ModelGiF: Gradient Fields for Model Functional Distance](https://mlsanity.com/paper/2309.11013/) (2023)
- [AWEncoder: Adversarial Watermarking Pre-trained Encoders in Contrastive Learning](https://mlsanity.com/paper/2208.03948/) (2022)
- [RemovalNet: DNN Fingerprint Removal Attacks](https://mlsanity.com/paper/2308.12319/) (2023)
- [Are You Copying My Prompt? Protecting the Copyright of Vision Prompt for VPaaS via Watermark](https://mlsanity.com/paper/2405.15161/) (2024)
- [Towards Open Federated Learning Platforms: Survey and Vision from Technical and Legal Perspectives](https://mlsanity.com/paper/2307.02140/) (2023)
- [Safe and Robust Watermark Injection with a Single OoD Image](https://mlsanity.com/paper/2309.01786/) (2023)
- [Watermarking Graph Neural Networks via Explanations for Ownership Protection](https://mlsanity.com/paper/2501.05614/) (2025)
- [AWM: Accurate Weight-Matrix Fingerprint for Large Language Models](https://mlsanity.com/paper/2510.06738/) (2025)
- [Knowledge-Free Black-Box Watermark and Ownership Proof for Image Classification Neural Networks](https://mlsanity.com/paper/2204.04522/) (2022)
- [A Comprehensive Study of Privacy Risks in Curriculum Learning](https://mlsanity.com/paper/2310.10124/) (2023)

## Abstract (arXiv)

Deep learning (DL) models, especially those large-scale and high-performance ones, can be very costly to train, demanding a great amount of data and computational resources. Unauthorized reproduction of DL models can lead to copyright infringement and cause huge economic losses to model owners. Existing copyright protection techniques are mostly based on watermarking, which embeds an owner-specified watermark into the model. While being able to provide exact ownership verification, these techniques are 1) invasive, as they need to tamper with the training process, which may affect the utility or introduce new security risks; 2) prone to adaptive attacks that attempt to remove the watermark; and 3) not robust to the emerging model extraction attacks. Latest fingerprinting work, though being non-invasive, also falls short when facing the diverse and ever-growing attack scenarios. In this paper, we propose a novel testing framework for DL copyright protection: DEEPJUDGE. DEEPJUDGE quantitatively tests the similarities between two DL models: a victim model and a suspect model. It leverages a diverse set of testing metrics and test case generation methods to produce a chain of supporting evidence to help determine whether a suspect model is a copy of the victim model. Advantages of DEEPJUDGE include: 1) non-invasive, as it works directly on the model and does not tamper with the training process; 2) efficient, as it only needs a small set of test cases and a quick scan of models; 3) flexible, as it can easily incorporate new metrics or generation methods to obtain more confident judgement; and 4) fairly robust to model extraction and adaptive attacks. We verify the effectiveness of DEEPJUDGE under typical copyright infringement scenarios, including model finetuning, pruning and extraction, via extensive experiments on both image and speech datasets with a variety of model architectures.