mlsanityData to 7 Oct 2026

SSL-WM: A Black-Box Watermarking Approach for Encoders Pre-trained by Self-supervised Learning

arXiv 2209.03563 ↗First posted 8 Sep 2022cs.CR cs.AIcited by 9

Not on the map. Why?

Abstract (arXiv)

Recent years have witnessed tremendous success in Self-Supervised Learning (SSL), which has been widely utilized to facilitate various downstream tasks in Computer Vision (CV) and Natural Language Processing (NLP) domains. However, attackers may steal such SSL models and commercialize them for profit, making it crucial to verify the ownership of the SSL models. Most existing ownership protection solutions (e.g., backdoor-based watermarks) are designed for supervised learning models and cannot be used directly since they require that the models' downstream tasks and target labels be known and available during watermark embedding, which is not always possible in the domain of SSL. To address such a problem, especially when downstream tasks are diverse and unknown during watermark embedding, we propose a novel black-box watermarking solution, named SSL-WM, for verifying the ownership of SSL models. SSL-WM maps watermarked inputs of the protected encoders into an invariant representation space, which causes any downstream classifier to produce expected behavior, thus allowing the detection of embedded watermarks. We evaluate SSL-WM on numerous tasks, such as CV and NLP, using different SSL models both contrastive-based and generative-based. Experimental results demonstrate that SSL-WM can effectively verify the ownership of stolen SSL models in various downstream tasks. Furthermore, SSL-WM is robust against model fine-tuning, pruning, and input preprocessing attacks. Lastly, SSL-WM can also evade detection from evaluated watermark detection approaches, demonstrating its promising application in protecting the ownership of SSL models.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 42 of its 62 references that are in the map. First posted on arXiv 8 Sep 2022.

Builds on

42 of 62 references.

References · 42

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  5. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  6. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  7. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  8. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  9. 2016Pointer Sentinel Mixture Modelscited by 2,992
  10. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,801
  11. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
  12. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,282
30 more
  1. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 1,998
  2. 2016node2vec: Scalable Feature Learning for Networkscited by 1,913
  3. 2016Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Networkcited by 1,898
  4. 2015A large annotated corpus for learning natural language inferencecited by 1,837
  5. 2016Context Encoders: Feature Learning by Inpaintingcited by 1,424
  6. 2016Variational Graph Auto-Encoderscited by 1,373
  7. 2016Conditional Image Generation with PixelCNN Decoderscited by 859
  8. 2019SpanBERT: Improving Pre-training by Representing and Predicting Spanscited by 832
  9. 2021Generalizing to Unseen Domains: A Survey on Domain Generalizationcited by 762
  10. 2016Adversarial Feature Learningcited by 350
  11. 2018Turning Your Weakness Into a Strength: Watermarking Deep Neural Networks by Backdooringcited by 334
  12. 2017Unified Deep Supervised Domain Adaptation and Generalizationcited by 322
  13. 2016Learning Representations for Automatic Colorizationcited by 251
  14. 2019Domain Randomization and Pyramid Consistency: Simulation-to-Real Generalization without Accessing Target Domain Datacited by 245
  15. 2019Large Scale Adversarial Representation Learningcited by 238
  16. 2017Embedding Watermarks into Deep Neural Networkscited by 227
  17. 2016Adversarially Learned Inferencecited by 216
  18. 2015Semi-supervised Sequence Learningcited by 198
  19. 2020Blind Backdoors in Deep Learning Modelscited by 174
  20. 2020Entangled Watermarks as a Defense against Model Extractioncited by 144
  21. 2021BadEncoder: Backdoor Attacks to Pre-trained Encoders in Self-Supervised Learningcited by 120
  22. 2021Backdoor Pre-trained Models Can Transfer to Allcited by 79
  23. 2019How to Prove Your Model Belongs to You: A Blind-Watermark based Framework to Protect Intellectual Property of DNNcited by 58
  24. 2019Robust Watermarking of Neural Network with Exponential Weightingcited by 48
  25. 2022SSLGuard: A Watermarking Scheme for Self-supervised Learning Pre-trained Encoderscited by 43
  26. 2018Stealing Neural Networks via Timing Side Channelscited by 42
  27. 2021Mosaicking to Distill: Knowledge Distillation from Out-of-Domain Datacited by 31
  28. 2018DeepSigns: A Generic Watermarking Framework for IP Protection of Deep Learning Modelscited by 16
  29. 2022Watermarking Pre-trained Encoders in Contrastive Learningcited by 10
  30. 2021HufuNet: Embedding the Left Piece as Watermark and Keeping the Right Piece for Ownership Verification in Deep Neural Networkscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 3; 2025: 1; 2026: 4.

9 papers.

  1. 2024SleeperMark: Towards Robust Watermark against Fine-Tuning Text-to-image Diffusion Modelscited by 26
  2. 2024MEA-Defender: A Robust Watermark against Model Extraction Attackcited by 16
  3. 2023SSL-Auth: An Authentication Framework by Fragile Watermarking for Pre-trained Encoders in Self-supervised Learningcited by 1
  4. 2024Pre-trained Encoder Inference: Revealing Upstream Encoders In Downstream Machine Learning Servicescited by 0
  5. 2025Neural Honeytrace: Plug&Play Watermarking Framework against Model Extraction Attackscited by 0
  6. 2026ArmSSL: Adversarial Robust Black-Box Watermarking for Self-Supervised Learning Pre-trained Encoderscited by 0
  7. 2026Repurposing and Evaluating the (In)Feasibility of Dataset Poisoning enabled Watermarking for Contrastive Learningcited by 0
  8. 2026MeMark: Membrane-Space Watermarking for Spiking Neural Networkscited by 0
  9. 2026Steal the Knowledge, Inherit the Mark: TwinMark for Distillation Watermarking via Second Moments and Class Multiplexingcited by 0