# SoK: How Robust is Image Classification Deep Neural Network Watermarking? (Extended Version)

arXiv: https://arxiv.org/abs/2108.04974 · HTML: https://mlsanity.com/paper/2108.04974/

## Key facts

- Cited by 69 papers in the map (data through 14 Oct 2026).
- Builds on 45 of its 72 references that are in the map.
- First posted on arXiv 11 Aug 2021.

## Builds on (most cited first, 30 of 45)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [Wide Residual Networks](https://mlsanity.com/paper/1605.07146/) (2016)
- [Identity Mappings in Deep Residual Networks](https://mlsanity.com/paper/1603.05027/) (2016)
- [Similarity of Neural Network Representations Revisited](https://mlsanity.com/paper/1905.00414/) (2019)
- [SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size](https://mlsanity.com/paper/1602.07360/) (2016)
- [CosFace: Large Margin Cosine Loss for Deep Face Recognition](https://mlsanity.com/paper/1801.09414/) (2018)
- [Big Transfer (BiT): General Visual Representation Learning](https://mlsanity.com/paper/1912.11370/) (2019)
- [SphereFace: Deep Hypersphere Embedding for Face Recognition](https://mlsanity.com/paper/1704.08063/) (2017)
- [Stealing Machine Learning Models via Prediction APIs](https://mlsanity.com/paper/1609.02943/) (2016)
- [To prune, or not to prune: exploring the efficacy of pruning for model compression](https://mlsanity.com/paper/1710.01878/) (2017)
- [Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networks](https://mlsanity.com/paper/1805.12185/) (2018)
- [Feature Squeezing: Detecting Adversarial Examples in Deep Neural Networks](https://mlsanity.com/paper/1704.01155/) (2017)
- [Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations](https://mlsanity.com/paper/1609.07061/) (2016)
- [Turning Your Weakness Into a Strength: Watermarking Deep Neural Networks by Backdooring](https://mlsanity.com/paper/1802.04633/) (2018)
- [Knockoff Nets: Stealing Functionality of Black-Box Models](https://mlsanity.com/paper/1812.02766/) (2018)
- [Embedding Watermarks into Deep Neural Networks](https://mlsanity.com/paper/1701.04082/) (2017)
- [High Accuracy and High Fidelity Extraction of Neural Networks](https://mlsanity.com/paper/1909.01838/) (2019)
- [A study of the effect of JPG compression on adversarial images](https://mlsanity.com/paper/1608.00853/) (2016)

## Built on by (most cited first, 30 of 69)

- [Leveraging Optimization for Adaptive Attacks on Image Watermarks](https://mlsanity.com/paper/2309.16952/) (2023)
- [SSLGuard: A Watermarking Scheme for Self-supervised Learning Pre-trained Encoders](https://mlsanity.com/paper/2201.11692/) (2022)
- [Certified Neural Network Watermarks with Randomized Smoothing](https://mlsanity.com/paper/2207.07972/) (2022)
- [Explanation as a Watermark: Towards Harmless and Multi-bit Model Ownership Verification via Watermarking Feature Attribution](https://mlsanity.com/paper/2405.04825/) (2024)
- [PTW: Pivotal Tuning Watermarking for Pre-Trained Image Generators](https://mlsanity.com/paper/2304.07361/) (2023)
- [Watermarking Graph Neural Networks based on Backdoor Attacks](https://mlsanity.com/paper/2110.11024/) (2021)
- [StolenEncoder: Stealing Pre-trained Encoders in Self-supervised Learning](https://mlsanity.com/paper/2201.05889/) (2022)
- [False Claims against Model Ownership Resolution](https://mlsanity.com/paper/2304.06607/) (2023)
- [MEA-Defender: A Robust Watermark against Model Extraction Attack](https://mlsanity.com/paper/2401.15239/) (2024)
- [GrOVe: Ownership Verification of Graph Neural Networks using Embeddings](https://mlsanity.com/paper/2304.08566/) (2023)
- [Machine Learning with Confidential Computing: A Systematization of Knowledge](https://mlsanity.com/paper/2208.10134/) (2022)
- [SoK: Large Language Model Copyright Auditing via Fingerprinting](https://mlsanity.com/paper/2508.19843/) (2025)
- [Conflicting Interactions Among Protection Mechanisms for Machine Learning Models](https://mlsanity.com/paper/2207.01991/) (2022)
- [What can Discriminator do? Towards Box-free Ownership Verification of Generative Adversarial Network](https://mlsanity.com/paper/2307.15860/) (2023)
- [Towards Robust Model Watermark via Reducing Parametric Vulnerability](https://mlsanity.com/paper/2309.04777/) (2023)
- [SoK: How Robust is Audio Watermarking in Generative AI models?](https://mlsanity.com/paper/2503.19176/) (2025)
- [Reliable Model Watermarking: Defending Against Theft without Compromising on Evasion](https://mlsanity.com/paper/2404.13518/) (2024)
- [QUEEN: Query Unlearning against Model Extraction](https://mlsanity.com/paper/2407.01251/) (2024)
- [Watermark under Fire: A Robustness Evaluation of LLM Watermarking](https://mlsanity.com/paper/2411.13425/) (2024)
- [On the Robustness of Dataset Inference](https://mlsanity.com/paper/2210.13631/) (2022)
- [DATABench: Evaluating Dataset Auditing in Deep Learning from an Adversarial Perspective](https://mlsanity.com/paper/2507.05622/) (2025)
- [When Federated Learning meets Watermarking: A Comprehensive Overview of Techniques for Intellectual Property Protection](https://mlsanity.com/paper/2308.03573/) (2023)
- [Removal Attack and Defense on AI-generated Content Latent-based Watermarking](https://mlsanity.com/paper/2509.11745/) (2025)
- [SoK: Unintended Interactions among Machine Learning Defenses and Risks](https://mlsanity.com/paper/2312.04542/) (2023)
- [Box-Free Model Watermarks Are Prone to Black-Box Removal Attacks](https://mlsanity.com/paper/2405.09863/) (2024)
- [Fostering the Robustness of White-Box Deep Neural Network Watermarks by Neuron Alignment](https://mlsanity.com/paper/2112.14108/) (2021)
- [DeepEclipse: How to Break White-Box DNN-Watermarking Schemes](https://mlsanity.com/paper/2403.03590/) (2024)
- [RemovalNet: DNN Fingerprint Removal Attacks](https://mlsanity.com/paper/2308.12319/) (2023)
- [Are You Copying My Prompt? Protecting the Copyright of Vision Prompt for VPaaS via Watermark](https://mlsanity.com/paper/2405.15161/) (2024)
- [On Evaluating The Performance of Watermarked Machine-Generated Texts Under Adversarial Attacks](https://mlsanity.com/paper/2407.04794/) (2024)

## Abstract (arXiv)

Deep Neural Network (DNN) watermarking is a method for provenance verification of DNN models. Watermarking should be robust against watermark removal attacks that derive a surrogate model that evades provenance verification. Many watermarking schemes that claim robustness have been proposed, but their robustness is only validated in isolation against a relatively small set of attacks. There is no systematic, empirical evaluation of these claims against a common, comprehensive set of removal attacks. This uncertainty about a watermarking scheme's robustness causes difficulty to trust their deployment in practice. In this paper, we evaluate whether recently proposed watermarking schemes that claim robustness are robust against a large set of removal attacks. We survey methods from the literature that (i) are known removal attacks, (ii) derive surrogate models but have not been evaluated as removal attacks, and (iii) novel removal attacks. Weight shifting and smooth retraining are novel removal attacks adapted to the DNN watermarking schemes surveyed in this paper. We propose taxonomies for watermarking schemes and removal attacks. Our empirical evaluation includes an ablation study over sets of parameters for each attack and watermarking scheme on the CIFAR-10 and ImageNet datasets. Surprisingly, none of the surveyed watermarking schemes is robust in practice. We find that schemes fail to withstand adaptive attacks and known methods for deriving surrogate models that have not been evaluated as removal attacks. This points to intrinsic flaws in how robustness is currently evaluated. We show that watermarking schemes need to be evaluated against a more extensive set of removal attacks with a more realistic adversary model. Our source code and a complete dataset of evaluation results are publicly available, which allows to independently verify our conclusions.