# ModelShield: Adaptive and Robust Watermark against Model Extraction Attack

arXiv: https://arxiv.org/abs/2405.02365 · HTML: https://mlsanity.com/paper/2405.02365/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Placed in Watermarking generated content and models, one of 12 directions in LLM safety and security.
- Builds on 42 of its 77 references that are in the map.
- First posted on arXiv 3 May 2024.

## Placed in

- LLM safety and security / [Watermarking generated content and models](https://mlsanity.com/safety/watermarking-generated-content-and-models/)

## Builds on (most cited first, 30 of 42)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Mistral 7B](https://mlsanity.com/paper/2310.06825/) (2023)
- [Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/1610.05820/) (2016)
- [Knowledge Distillation: A Survey](https://mlsanity.com/paper/2006.05525/) (2020)
- [A Watermark for Large Language Models](https://mlsanity.com/paper/2301.10226/) (2023)
- [Stealing Machine Learning Models via Prediction APIs](https://mlsanity.com/paper/1609.02943/) (2016)
- [GoEmotions: A Dataset of Fine-Grained Emotions](https://mlsanity.com/paper/2005.00547/) (2020)
- [How Close is ChatGPT to Human Experts? Comparison Corpus, Evaluation, and Detection](https://mlsanity.com/paper/2301.07597/) (2023)
- [Robust Distortion-free Watermarks for Language Models](https://mlsanity.com/paper/2307.15593/) (2023)
- [Undetectable Watermarks for Language Models](https://mlsanity.com/paper/2306.09194/) (2023)
- [High Accuracy and High Fidelity Extraction of Neural Networks](https://mlsanity.com/paper/1909.01838/) (2019)
- [PRADA: Protecting against DNN Model Stealing Attacks](https://mlsanity.com/paper/1805.02628/) (2018)
- [Mind the Style of Text! Adversarial and Backdoor Attacks Based on Text Style Transfer](https://mlsanity.com/paper/2110.07139/) (2021)
- [Thieves on Sesame Street! Model Extraction of BERT-based APIs](https://mlsanity.com/paper/1910.12366/) (2019)
- [Hidden Backdoors in Human-Centric Language Models](https://mlsanity.com/paper/2105.00164/) (2021)
- [Protecting Language Generation Models via Invisible Watermarking](https://mlsanity.com/paper/2302.03162/) (2023)
- [Untargeted Backdoor Watermark: Towards Harmless and Stealthy Dataset Copyright Protection](https://mlsanity.com/paper/2210.00875/) (2022)
- [Protecting Intellectual Property of Language Generation APIs with Lexical Watermark](https://mlsanity.com/paper/2112.02701/) (2021)
- [Prediction Poisoning: Towards Defenses Against DNN Model Stealing Attacks](https://mlsanity.com/paper/1906.10908/) (2019)
- [Imitation Attacks and Defenses for Black-box Machine Translation Systems](https://mlsanity.com/paper/2004.15015/) (2020)
- [CATER: Intellectual Property Protection on Text Generation APIs via Conditional Watermarks](https://mlsanity.com/paper/2209.08773/) (2022)
- [Publicly-Detectable Watermarking for Language Models](https://mlsanity.com/paper/2310.18491/) (2023)
- [Are You Copying My Model? Protecting the Copyright of Large Language Models for EaaS via Backdoor Watermark](https://mlsanity.com/paper/2305.10036/) (2023)
- [Black-box Dataset Ownership Verification via Backdoor Watermarking](https://mlsanity.com/paper/2209.06015/) (2022)

## Built on by (most cited first, 7 of 7)

- [FIT-Print: Towards False-claim-resistant Model Ownership Verification via Targeted Fingerprint](https://mlsanity.com/paper/2501.15509/) (2025)
- [Watermarking Language Models through Language Models](https://mlsanity.com/paper/2411.05091/) (2024)
- [MISLEADER: Defending against Model Extraction with Ensembles of Distilled Models](https://mlsanity.com/paper/2506.02362/) (2025)
- [Robust LLM Watermarking with Minimal Semantic Distortion for IP Protection](https://mlsanity.com/paper/2605.23175/) (2026)
- [LLM Watermarking as Big Data Provenance: A Deployment-Oriented Systematization](https://mlsanity.com/paper/2607.10103/) (2026)
- [FLClear: Visually Verifiable Multi-Client Watermarking for Federated Learning](https://mlsanity.com/paper/2511.12663/) (2025)
- [R-CoT: A Reasoning-Layer Watermark via Redundant Chain-of-Thought in Large Language Models](https://mlsanity.com/paper/2604.25247/) (2026)

## Abstract (arXiv)

Large language models (LLMs) demonstrate general intelligence across a variety of machine learning tasks, thereby enhancing the commercial value of their intellectual property (IP). To protect this IP, model owners typically allow user access only in a black-box manner, however, adversaries can still utilize model extraction attacks to steal the model intelligence encoded in model generation. Watermarking technology offers a promising solution for defending against such attacks by embedding unique identifiers into the model-generated content. However, existing watermarking methods often compromise the quality of generated content due to heuristic alterations and lack robust mechanisms to counteract adversarial strategies, thus limiting their practicality in real-world scenarios. In this paper, we introduce an adaptive and robust watermarking method (named ModelShield) to protect the IP of LLMs. Our method incorporates a self-watermarking mechanism that allows LLMs to autonomously insert watermarks into their generated content to avoid the degradation of model content. We also propose a robust watermark detection mechanism capable of effectively identifying watermark signals under the interference of varying adversarial strategies. Besides, ModelShield is a plug-and-play method that does not require additional model training, enhancing its applicability in LLM deployments. Extensive evaluations on two real-world datasets and three LLMs demonstrate that our method surpasses existing methods in terms of defense effectiveness and robustness while significantly reducing the degradation of watermarking on the model-generated content.