# PromptCARE: Prompt Copyright Protection by Watermark Injection and Verification

arXiv: https://arxiv.org/abs/2308.02816 · HTML: https://mlsanity.com/paper/2308.02816/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Aug 2023.

## Built on by (most cited first, 30 of 32)

- [Safeguarding Large Language Models: A Survey](https://mlsanity.com/paper/2406.02622/) (2024)
- [Building Guardrails for Large Language Models](https://mlsanity.com/paper/2402.01822/) (2024)
- [Navigating the Risks: A Survey of Security, Privacy, and Ethics Threats in LLM-Based Agents](https://mlsanity.com/paper/2411.09523/) (2024)
- [PoisonPrompt: Backdoor Attack on Prompt-based Large Language Models](https://mlsanity.com/paper/2310.12439/) (2023)
- [Explanation as a Watermark: Towards Harmless and Multi-bit Model Ownership Verification via Watermarking Feature Attribution](https://mlsanity.com/paper/2405.04825/) (2024)
- [PR-Attack: Coordinated Prompt-RAG Attacks on Retrieval-Augmented Generation in Large Language Models via Bilevel Optimization](https://mlsanity.com/paper/2504.07717/) (2025)
- [Deconstructing The Ethics of Large Language Models from Long-standing Issues to New-emerging Dilemmas: A Survey](https://mlsanity.com/paper/2406.05392/) (2024)
- [Rethinking Data Protection in the (Generative) Artificial Intelligence Era](https://mlsanity.com/paper/2507.03034/) (2025)
- [SoK: Dataset Copyright Auditing in Machine Learning Systems](https://mlsanity.com/paper/2410.16618/) (2024)
- [ControlNET: A Firewall for RAG-based LLM System](https://mlsanity.com/paper/2504.09593/) (2025)
- [PRSA: Prompt Stealing Attacks against Real-World Prompt Services](https://mlsanity.com/paper/2402.19200/) (2024)
- [FIT-Print: Towards False-claim-resistant Model Ownership Verification via Targeted Fingerprint](https://mlsanity.com/paper/2501.15509/) (2025)
- [Watermarking Language Models through Language Models](https://mlsanity.com/paper/2411.05091/) (2024)
- [StyleGuard: Preventing Text-to-Image-Model-based Style Mimicry Attacks by Style Perturbations](https://mlsanity.com/paper/2505.18766/) (2025)
- [Prompt Engineering-assisted Malware Dynamic Analysis Using GPT-4](https://mlsanity.com/paper/2312.08317/) (2023)
- [Privacy-Preserving Personalized Federated Prompt Learning for Multimodal Large Language Models](https://mlsanity.com/paper/2501.13904/) (2025)
- [RemovalNet: DNN Fingerprint Removal Attacks](https://mlsanity.com/paper/2308.12319/) (2023)
- [Are You Copying My Prompt? Protecting the Copyright of Vision Prompt for VPaaS via Watermark](https://mlsanity.com/paper/2405.15161/) (2024)
- [Robust and Minimally Invasive Watermarking for EaaS](https://mlsanity.com/paper/2410.17552/) (2024)
- [PromptCOS: Towards Content-only System Prompt Copyright Auditing for LLMs](https://mlsanity.com/paper/2509.03117/) (2025)
- [Responsible Diffusion: A Comprehensive Survey on Safety, Ethics, and Trust in Diffusion Models](https://mlsanity.com/paper/2509.22723/) (2025)
- [Beyond Dataset Watermarking: Model-Level Copyright Protection for Code Summarization Models](https://mlsanity.com/paper/2410.14102/) (2024)
- [Towards Effective Prompt Stealing Attack against Text-to-Image Diffusion Models](https://mlsanity.com/paper/2508.06837/) (2025)
- [An Empirically-grounded tool for Automatic Prompt Linting and Repair: A Case Study on Bias, Vulnerability, and Optimization in Developer Prompts](https://mlsanity.com/paper/2501.12521/) (2025)
- [Safeguarding Multimodal Knowledge Copyright in the RAG-as-a-Service Environment](https://mlsanity.com/paper/2506.10030/) (2025)
- [PIDP-Attack: Combining Prompt Injection with Database Poisoning Attacks on Retrieval-Augmented Generation Systems](https://mlsanity.com/paper/2603.25164/) (2026)
- [PoLO: Proof-of-Learning and Proof-of-Ownership at Once with Chained Watermarking](https://mlsanity.com/paper/2505.12296/) (2025)
- ["I made this (sort of)": Negotiating authorship, confronting fraudulence, and exploring new musical spaces with prompt-based AI music generation](https://mlsanity.com/paper/2507.23365/) (2025)
- [User Negotiations of Authenticity, Ownership, and Governance on AI-Generated Video Platforms: Evidence from Sora](https://mlsanity.com/paper/2512.05519/) (2025)
- [Functional Subspace Watermarking for Large Language Models](https://mlsanity.com/paper/2603.18793/) (2026)

## Abstract (arXiv)

Large language models (LLMs) have witnessed a meteoric rise in popularity among the general public users over the past few months, facilitating diverse downstream tasks with human-level accuracy and proficiency. Prompts play an essential role in this success, which efficiently adapt pre-trained LLMs to task-specific applications by simply prepending a sequence of tokens to the query texts. However, designing and selecting an optimal prompt can be both expensive and demanding, leading to the emergence of Prompt-as-a-Service providers who profit by providing well-designed prompts for authorized use. With the growing popularity of prompts and their indispensable role in LLM-based services, there is an urgent need to protect the copyright of prompts against unauthorized use. In this paper, we propose PromptCARE, the first framework for prompt copyright protection through watermark injection and verification. Prompt watermarking presents unique challenges that render existing watermarking techniques developed for model and dataset copyright verification ineffective. PromptCARE overcomes these hurdles by proposing watermark injection and verification schemes tailor-made for prompts and NLP characteristics. Extensive experiments on six well-known benchmark datasets, using three prevalent pre-trained LLMs (BERT, RoBERTa, and Facebook OPT-1.3b), demonstrate the effectiveness, harmlessness, robustness, and stealthiness of PromptCARE.