# Are You Copying My Prompt? Protecting the Copyright of Vision Prompt for VPaaS via Watermark

arXiv: https://arxiv.org/abs/2405.15161 · HTML: https://mlsanity.com/paper/2405.15161/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 51 references that are in the map.
- First posted on arXiv 24 May 2024.

## Builds on (most cited first, 30 of 36)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://mlsanity.com/paper/2101.00190/) (2021)
- [The Power of Scale for Parameter-Efficient Prompt Tuning](https://mlsanity.com/paper/2104.08691/) (2021)
- [Visual Prompt Tuning](https://mlsanity.com/paper/2203.12119/) (2022)
- [A Comprehensive Survey on Transfer Learning](https://mlsanity.com/paper/1911.02685/) (2019)
- [Big Transfer (BiT): General Visual Representation Learning](https://mlsanity.com/paper/1912.11370/) (2019)
- [A Watermark for Large Language Models](https://mlsanity.com/paper/2301.10226/) (2023)
- [MaPLe: Multi-modal Prompt Learning](https://mlsanity.com/paper/2210.03117/) (2022)
- [Exploring the Limits of Weakly Supervised Pretraining](https://mlsanity.com/paper/1805.00932/) (2018)
- [Turning Your Weakness Into a Strength: Watermarking Deep Neural Networks by Backdooring](https://mlsanity.com/paper/1802.04633/) (2018)
- [Invisible Backdoor Attack with Sample-Specific Triggers](https://mlsanity.com/paper/2012.03816/) (2020)
- [Exploring Visual Prompts for Adapting Large-Scale Models](https://mlsanity.com/paper/2203.17274/) (2022)
- [Visual Prompting via Image Inpainting](https://mlsanity.com/paper/2209.00647/) (2022)
- [Embedding Watermarks into Deep Neural Networks](https://mlsanity.com/paper/1701.04082/) (2017)
- [Adversarial Reprogramming of Neural Networks](https://mlsanity.com/paper/1806.11146/) (2018)
- [Dataset Inference: Ownership Resolution in Machine Learning](https://mlsanity.com/paper/2104.10706/) (2021)
- [Visual Prompt Tuning for Test-time Domain Adaptation](https://mlsanity.com/paper/2210.04831/) (2022)
- [Understanding and Improving Visual Prompting: A Label-Mapping Perspective](https://mlsanity.com/paper/2211.11635/) (2022)
- [Untargeted Backdoor Watermark: Towards Harmless and Stealthy Dataset Copyright Protection](https://mlsanity.com/paper/2210.00875/) (2022)
- [Transfer Learning without Knowing: Reprogramming Black-box Machine Learning Models with Scarce Data and Limited Resources](https://mlsanity.com/paper/2007.08714/) (2020)
- [Visual Prompt Tuning for Generative Transfer Learning](https://mlsanity.com/paper/2210.00990/) (2022)
- [A survey of deep neural network watermarking techniques](https://mlsanity.com/paper/2103.09274/) (2021)
- [SoK: How Robust is Image Classification Deep Neural Network Watermarking? (Extended Version)](https://mlsanity.com/paper/2108.04974/) (2021)
- [Black-box Dataset Ownership Verification via Backdoor Watermarking](https://mlsanity.com/paper/2209.06015/) (2022)
- [Diversity-Aware Meta Visual Prompting](https://mlsanity.com/paper/2303.08138/) (2023)
- [Prompt Stealing Attacks Against Text-to-Image Generation Models](https://mlsanity.com/paper/2302.09923/) (2023)
- [Copy, Right? A Testing Framework for Copyright Protection of Deep Learning Models](https://mlsanity.com/paper/2112.05588/) (2021)
- [Dataset Inference for Self-Supervised Models](https://mlsanity.com/paper/2209.09024/) (2022)
- [Cross-modal Adversarial Reprogramming](https://mlsanity.com/paper/2102.07325/) (2021)

## Built on by (most cited first, 4 of 4)

- [Visual Prompting in Multimodal Large Language Models: A Survey](https://mlsanity.com/paper/2409.15310/) (2024)
- [Robust and Minimally Invasive Watermarking for EaaS](https://mlsanity.com/paper/2410.17552/) (2024)
- [Vision-Language Model IP Protection via Prompt-based Learning](https://mlsanity.com/paper/2503.02393/) (2025)
- [FlexICL: A Flexible Visual In-context Learning Framework for Elbow and Wrist Ultrasound Segmentation](https://mlsanity.com/paper/2510.26049/) (2025)

## Abstract (arXiv)

Visual Prompt Learning (VPL) differs from traditional fine-tuning methods in reducing significant resource consumption by avoiding updating pre-trained model parameters. Instead, it focuses on learning an input perturbation, a visual prompt, added to downstream task data for making predictions. Since learning generalizable prompts requires expert design and creation, which is technically demanding and time-consuming in the optimization process, developers of Visual Prompts as a Service (VPaaS) have emerged. These developers profit by providing well-crafted prompts to authorized customers. However, a significant drawback is that prompts can be easily copied and redistributed, threatening the intellectual property of VPaaS developers. Hence, there is an urgent need for technology to protect the rights of VPaaS developers. To this end, we present a method named WVPrompt that employs visual prompt watermarking in a black-box way. WVPrompt consists of two parts: prompt watermarking and prompt verification. Specifically, it utilizes a poison-only backdoor attack method to embed a watermark into the prompt and then employs a hypothesis-testing approach for remote verification of prompt ownership. Extensive experiments have been conducted on three well-known benchmark datasets using three popular pre-trained models: RN50, BIT-M, and Instagram. The experimental results demonstrate that WVPrompt is efficient, harmless, and robust to various adversarial operations.