# SSLGuard: A Watermarking Scheme for Self-supervised Learning Pre-trained Encoders

arXiv: https://arxiv.org/abs/2201.11692 · HTML: https://mlsanity.com/paper/2201.11692/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- Builds on 46 of its 63 references that are in the map.
- First posted on arXiv 27 Jan 2022.

## Builds on (most cited first, 30 of 46)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Improved Baselines with Momentum Contrastive Learning](https://mlsanity.com/paper/2003.04297/) (2020)
- [Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/1610.05820/) (2016)
- [Adversarial examples in the physical world](https://mlsanity.com/paper/1607.02533/) (2016)
- [Group Normalization](https://mlsanity.com/paper/1803.08494/) (2018)
- [Graph Contrastive Learning with Augmentations](https://mlsanity.com/paper/2010.13902/) (2020)
- [Pruning Filters for Efficient ConvNets](https://mlsanity.com/paper/1608.08710/) (2016)
- [Practical Black-Box Attacks against Machine Learning](https://mlsanity.com/paper/1602.02697/) (2016)
- [Stealing Machine Learning Models via Prediction APIs](https://mlsanity.com/paper/1609.02943/) (2016)
- [To prune, or not to prune: exploring the efficacy of pruning for model compression](https://mlsanity.com/paper/1710.01878/) (2017)
- [Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networks](https://mlsanity.com/paper/1805.12185/) (2018)
- [Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methods](https://mlsanity.com/paper/1705.07263/) (2017)
- [Delving into Transferable Adversarial Examples and Black-box Attacks](https://mlsanity.com/paper/1611.02770/) (2016)
- [ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Models](https://mlsanity.com/paper/1806.01246/) (2018)
- [Hidden Trigger Backdoor Attacks](https://mlsanity.com/paper/1910.00033/) (2019)
- [Turning Your Weakness Into a Strength: Watermarking Deep Neural Networks by Backdooring](https://mlsanity.com/paper/1802.04633/) (2018)
- [Knockoff Nets: Stealing Functionality of Black-Box Models](https://mlsanity.com/paper/1812.02766/) (2018)
- [Systematic Evaluation of Privacy Risks of Machine Learning Models](https://mlsanity.com/paper/2003.10595/) (2020)
- [DeCLUTR: Deep Contrastive Learning for Unsupervised Textual Representations](https://mlsanity.com/paper/2006.03659/) (2020)
- [Embedding Watermarks into Deep Neural Networks](https://mlsanity.com/paper/1701.04082/) (2017)
- [High Accuracy and High Fidelity Extraction of Neural Networks](https://mlsanity.com/paper/1909.01838/) (2019)

## Built on by (most cited first, 30 of 45)

- [AdvCLIP: Downstream-agnostic Adversarial Examples in Multimodal Contrastive Learning](https://mlsanity.com/paper/2308.07026/) (2023)
- [Mark My Words: Analyzing and Evaluating Language Model Watermarks](https://mlsanity.com/paper/2312.00273/) (2023)
- [Watermarking Pre-trained Language Models with Backdooring](https://mlsanity.com/paper/2210.07543/) (2022)
- [Explanation as a Watermark: Towards Harmless and Multi-bit Model Ownership Verification via Watermarking Feature Attribution](https://mlsanity.com/paper/2405.04825/) (2024)
- [Watermarking Diffusion Model](https://mlsanity.com/paper/2305.12502/) (2023)
- [Dataset Inference for Self-Supervised Models](https://mlsanity.com/paper/2209.09024/) (2022)
- [Downstream-agnostic Adversarial Examples](https://mlsanity.com/paper/2307.12280/) (2023)
- [Transferable Embedding Inversion Attack: Uncovering Privacy Risks in Text Embeddings without Model Queries](https://mlsanity.com/paper/2406.10280/) (2024)
- [Have You Merged My Model? On The Robustness of Large Language Model IP Protection Methods Against Model Merging](https://mlsanity.com/paper/2404.05188/) (2024)
- [MEA-Defender: A Robust Watermark against Model Extraction Attack](https://mlsanity.com/paper/2401.15239/) (2024)
- [Quantized Delta Weight Is Safety Keeper](https://mlsanity.com/paper/2411.19530/) (2024)
- [SSL-WM: A Black-Box Watermarking Approach for Encoders Pre-trained by Self-supervised Learning](https://mlsanity.com/paper/2209.03563/) (2022)
- [A Unified Membership Inference Method for Visual Self-supervised Encoder via Part-aware Capability](https://mlsanity.com/paper/2404.02462/) (2024)
- [QUEEN: Query Unlearning against Model Extraction](https://mlsanity.com/paper/2407.01251/) (2024)
- [Character-Level Perturbations Disrupt LLM Watermarks](https://mlsanity.com/paper/2509.09112/) (2025)
- [Bucks for Buckets (B4B): Active Defenses Against Stealing Encoders](https://mlsanity.com/paper/2310.08571/) (2023)
- [ImF: Implicit Fingerprint for Large Language Models](https://mlsanity.com/paper/2503.21805/) (2025)
- [Beyond Boundaries: A Comprehensive Survey of Transferable Attacks on AI Systems](https://mlsanity.com/paper/2311.11796/) (2023)
- [A Plot is Worth a Thousand Words: Model Information Stealing Attacks via Scientific Plots](https://mlsanity.com/paper/2302.11982/) (2023)
- [DeepEclipse: How to Break White-Box DNN-Watermarking Schemes](https://mlsanity.com/paper/2403.03590/) (2024)
- ["Yes, My LoRD." Guiding Language Model Extraction with Locality Reinforced Distillation](https://mlsanity.com/paper/2409.02718/) (2024)
- [AWEncoder: Adversarial Watermarking Pre-trained Encoders in Contrastive Learning](https://mlsanity.com/paper/2208.03948/) (2022)
- [On Evaluating The Performance of Watermarked Machine-Generated Texts Under Adversarial Attacks](https://mlsanity.com/paper/2407.04794/) (2024)
- [An Empirical Study on Large Language Models in Accuracy and Robustness under Chinese Industrial Scenarios](https://mlsanity.com/paper/2402.01723/) (2024)
- [PreGIP: Watermarking the Pretraining of Graph Neural Networks for Deep Intellectual Property Protection](https://mlsanity.com/paper/2402.04435/) (2024)
- [Revisiting the Information Capacity of Neural Network Watermarks: Upper Bound Estimation and Beyond](https://mlsanity.com/paper/2402.12720/) (2024)
- [Intellectual Property in Graph-Based Machine Learning as a Service: Attacks and Defenses](https://mlsanity.com/paper/2508.19641/) (2025)
- [A Comprehensive Study of Privacy Risks in Curriculum Learning](https://mlsanity.com/paper/2310.10124/) (2023)
- [BESA: Boosting Encoder Stealing Attack with Perturbation Recovery](https://mlsanity.com/paper/2506.04556/) (2025)
- [Towards Backdoor-Based Ownership Verification for Vision-Language-Action Models](https://mlsanity.com/paper/2605.09005/) (2026)

## Abstract (arXiv)

Self-supervised learning is an emerging machine learning paradigm. Compared to supervised learning which leverages high-quality labeled datasets, self-supervised learning relies on unlabeled datasets to pre-train powerful encoders which can then be treated as feature extractors for various downstream tasks. The huge amount of data and computational resources consumption makes the encoders themselves become the valuable intellectual property of the model owner. Recent research has shown that the machine learning model's copyright is threatened by model stealing attacks, which aim to train a surrogate model to mimic the behavior of a given model. We empirically show that pre-trained encoders are highly vulnerable to model stealing attacks. However, most of the current efforts of copyright protection algorithms such as watermarking concentrate on classifiers. Meanwhile, the intrinsic challenges of pre-trained encoder's copyright protection remain largely unstudied. We fill the gap by proposing SSLGuard, the first watermarking scheme for pre-trained encoders. Given a clean pre-trained encoder, SSLGuard injects a watermark into it and outputs a watermarked version. The shadow training technique is also applied to preserve the watermark under potential model stealing attacks. Our extensive evaluation shows that SSLGuard is effective in watermark injection and verification, and it is robust against model stealing and other watermark removal attacks such as input noising, output perturbing, overwriting, model pruning, and fine-tuning.