# WARDEN: Multi-Directional Backdoor Watermarks for Embedding-as-a-Service Copyright Protection

arXiv: https://arxiv.org/abs/2403.01472 · HTML: https://mlsanity.com/paper/2403.01472/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 47 references that are in the map.
- First posted on arXiv 3 Mar 2024.

## Builds on (most cited first, 25 of 25)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Pointer Sentinel Mixture Models](https://mlsanity.com/paper/1609.07843/) (2016)
- [Character-level Convolutional Networks for Text Classification](https://mlsanity.com/paper/1509.01626/) (2015)
- [Stealing Machine Learning Models via Prediction APIs](https://mlsanity.com/paper/1609.02943/) (2016)
- [Generating Natural Language Adversarial Examples](https://mlsanity.com/paper/1804.07998/) (2018)
- [Knockoff Nets: Stealing Functionality of Black-Box Models](https://mlsanity.com/paper/1812.02766/) (2018)
- [Embedding Watermarks into Deep Neural Networks](https://mlsanity.com/paper/1701.04082/) (2017)
- [A backdoor attack against LSTM-based text classification systems](https://mlsanity.com/paper/1905.12457/) (2019)
- [PRADA: Protecting against DNN Model Stealing Attacks](https://mlsanity.com/paper/1805.02628/) (2018)
- [Thieves on Sesame Street! Model Extraction of BERT-based APIs](https://mlsanity.com/paper/1910.12366/) (2019)
- [Exploring Connections Between Active Learning and Model Extraction](https://mlsanity.com/paper/1811.02054/) (2018)
- [BadPre: Task-agnostic Backdoor Attacks to Pre-trained NLP Foundation Models](https://mlsanity.com/paper/2110.02467/) (2021)
- [Protecting Intellectual Property of Language Generation APIs with Lexical Watermark](https://mlsanity.com/paper/2112.02701/) (2021)
- [Imitation Attacks and Defenses for Black-box Machine Translation Systems](https://mlsanity.com/paper/2004.15015/) (2020)
- [CATER: Intellectual Property Protection on Text Generation APIs via Conditional Watermarks](https://mlsanity.com/paper/2209.08773/) (2022)
- [Are You Copying My Model? Protecting the Copyright of Large Language Models for EaaS via Backdoor Watermark](https://mlsanity.com/paper/2305.10036/) (2023)
- [Model Extraction and Adversarial Transferability, Your BERT is Vulnerable!](https://mlsanity.com/paper/2103.10013/) (2021)
- [Red Alarm for Pre-trained Models: Universal Vulnerability to Neuron-Level Backdoor Attacks](https://mlsanity.com/paper/2101.06969/) (2021)
- [Black-Box Attacks on Sequential Recommenders via Data-Free Model Extraction](https://mlsanity.com/paper/2109.01165/) (2021)
- [Training-free Lexical Backdoor Attacks on Language Models](https://mlsanity.com/paper/2302.04116/) (2023)
- [Protect, Show, Attend and Tell: Empowering Image Captioning Models with Ownership Protection](https://mlsanity.com/paper/2008.11009/) (2020)
- [Student Surpasses Teacher: Imitation Attack for Black-Box NLP APIs](https://mlsanity.com/paper/2108.13873/) (2021)
- [Watermarking Vision-Language Pre-trained Models for Multi-modal Embedding as a Service](https://mlsanity.com/paper/2311.05863/) (2023)

## Built on by (most cited first, 12 of 12)

- [Watermarking Techniques for Large Language Models: A Survey](https://mlsanity.com/paper/2409.00089/) (2024)
- [ModelShield: Adaptive and Robust Watermark against Model Extraction Attack](https://mlsanity.com/paper/2405.02365/) (2024)
- [WET: Overcoming Paraphrasing Vulnerabilities in Embeddings-as-a-Service with Linear Transformation Watermarks](https://mlsanity.com/paper/2409.04459/) (2024)
- [Robust and Minimally Invasive Watermarking for EaaS](https://mlsanity.com/paper/2410.17552/) (2024)
- [RegionMarker: A Region-Triggered Semantic Watermarking Framework for Embedding-as-a-Service Copyright Protection](https://mlsanity.com/paper/2511.13329/) (2025)
- [From Essence to Defense: Adaptive Semantic-aware Watermarking for Embedding-as-a-Service Copyright Protection](https://mlsanity.com/paper/2512.16439/) (2025)
- [EmbTracker: Traceable Black-box Watermarking for Federated Language Models](https://mlsanity.com/paper/2603.12089/) (2026)
- [Protecting Copyright of Medical Pre-trained Language Models: Training-Free Backdoor Model Watermarking](https://mlsanity.com/paper/2409.10570/) (2024)
- [Your Semantic-Independent Watermark is Fragile: A Semantic Perturbation Attack against EaaS Watermark](https://mlsanity.com/paper/2411.09359/) (2024)
- [Rotation, Scale, and Translation Resilient Black-box Fingerprinting for Intellectual Property Protection of EaaS Models](https://mlsanity.com/paper/2510.16706/) (2025)
- [Geometry-Aware Localized Watermarking for Copyright Protection in Embedding-as-a-Service](https://mlsanity.com/paper/2604.11344/) (2026)
- [ArmSSL: Adversarial Robust Black-Box Watermarking for Self-Supervised Learning Pre-trained Encoders](https://mlsanity.com/paper/2604.22550/) (2026)

## Abstract (arXiv)

Embedding as a Service (EaaS) has become a widely adopted solution, which offers feature extraction capabilities for addressing various downstream tasks in Natural Language Processing (NLP). Prior studies have shown that EaaS can be prone to model extraction attacks; nevertheless, this concern could be mitigated by adding backdoor watermarks to the text embeddings and subsequently verifying the attack models post-publication. Through the analysis of the recent watermarking strategy for EaaS, EmbMarker, we design a novel CSE (Clustering, Selection, Elimination) attack that removes the backdoor watermark while maintaining the high utility of embeddings, indicating that the previous watermarking approach can be breached. In response to this new threat, we propose a new protocol to make the removal of watermarks more challenging by incorporating multiple possible watermark directions. Our defense approach, WARDEN, notably increases the stealthiness of watermarks and has been empirically shown to be effective against CSE attack.