mlsanityData to 7 Oct 2026

Prompt Backdoors in Visual Prompt Learning

arXiv 2310.07632 ↗First posted 11 Oct 2023cs.CR cs.CV cs.LGcited by 4

On the map: LLM safety and security → Backdoor and data poisoning attacks

Abstract (arXiv)

Fine-tuning large pre-trained computer vision models is infeasible for resource-limited users. Visual prompt learning (VPL) has thus emerged to provide an efficient and flexible alternative to model fine-tuning through Visual Prompt as a Service (VPPTaaS). Specifically, the VPPTaaS provider optimizes a visual prompt given downstream data, and downstream users can use this prompt together with the large pre-trained model for prediction. However, this new learning paradigm may also pose security risks when the VPPTaaS provider instead provides a malicious visual prompt. In this paper, we take the first step to explore such risks through the lens of backdoor attacks. Specifically, we propose BadVisualPrompt, a simple yet effective backdoor attack against VPL. For example, poisoning 5% CIFAR10 training data leads to above 99% attack success rates with only negligible model accuracy drop by 1.5%. In particular, we identify and then address a new technical challenge related to interactions between the backdoor trigger and visual prompt, which does not exist in conventional, model-level backdoors. Moreover, we provide in-depth analyses of seven backdoor defenses from model, prompt, and input levels. Overall, all these defenses are either ineffective or impractical to mitigate our BadVisualPrompt, implying the critical vulnerability of VPL.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in Backdoor and data poisoning attacks, one of 12 directions in LLM safety and security. Builds on 59 of its 77 references that are in the map. First posted on arXiv 11 Oct 2023.

Builds on

59 of 77 references.

Closest ancestors · 3

Same direction, newest first

  1. 2022BadPrompt: Backdoor Attacks on Continuous Prompts27 Nov
  2. 2021BadEncoder: Backdoor Attacks to Pre-trained Encoders in Self-Supervised Learning1 Aug
  3. 2021Sleeper Agent: Scalable Hidden Trigger Backdoors for Neural Networks Trained from Scratch16 Jun

Other references · 56

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  5. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  6. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  7. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  8. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
  9. 2021Learning to Prompt for Vision-Language Modelscited by 2,269
  10. 2019Language Models as Knowledge Bases?cited by 1,922
  11. 2022Visual Prompt Tuningcited by 1,674
  12. 2019A Comprehensive Survey on Transfer Learningcited by 1,154
38 more
  1. 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,122
  2. 2017BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chaincited by 1,052
  3. 2019How Can We Know What Language Models Know?cited by 953
  4. 2019Big Transfer (BiT): General Visual Representation Learningcited by 766
  5. 2018Exploring the Limits of Weakly Supervised Pretrainingcited by 604
  6. 2021Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigmcited by 599
  7. 2018Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networkscited by 550
  8. 2017Adversarial Patchcited by 514
  9. 2019STRIP: A Defence Against Trojan Attacks on Deep Neural Networkscited by 437
  10. 2018Detecting Backdoor Attacks on Deep Neural Networks by Activation Clusteringcited by 429
  11. 2019Hidden Trigger Backdoor Attackscited by 359
  12. 2020Attack of the Tails: Yes, You Really Can Backdoor Federated Learningcited by 307
  13. 2022Exploring Visual Prompts for Adapting Large-Scale Modelscited by 271
  14. 2021PTR: Prompt Tuning with Rules for Text Classificationcited by 238
  15. 2021WARP: Word-level Adversarial ReProgrammingcited by 220
  16. 2019Detecting AI Trojans Using Meta Neural Analysiscited by 192
  17. 2019Commonsense Knowledge Mining from Pretrained Modelscited by 189
  18. 2021You Only Look at One Sequence: Rethinking Transformer in Vision through Object Detectioncited by 170
  19. 2020Clean-Label Backdoor Attacks on Video Recognition Modelscited by 162
  20. 2020Witches' Brew: Industrial Scale Data Poisoning via Gradient Matchingcited by 156
  21. 2021OpenPrompt: An Open-source Framework for Prompt-learningcited by 148
  22. 2022Data Determines Distributional Robustness in Contrastive Language Image Pre-training (CLIP)cited by 137
  23. 2019Demon in the Variant: Statistical Analysis of DNNs for Robust Backdoor Contamination Detectioncited by 121
  24. 2020MetaPoison: Practical General-purpose Clean-label Data Poisoningcited by 114
  25. 2022Narcissus: A Practical Clean-Label Backdoor Attack with Limited Informationcited by 102
  26. 2020Automatically Identifying Words That Can Serve as Labels for Few-Shot Text Classificationcited by 98
  27. 2022Decorate the Newcomers: Visual Domain Prompt for Continual Test Time Adaptationcited by 98
  28. 2022Quality Not Quantity: On the Interaction between Dataset Design and Robustness of CLIPcited by 97
  29. 2022Visual Prompt Tuning for Test-time Domain Adaptationcited by 97
  30. 2021Backdoor Attacks on Self-Supervised Learningcited by 89
  31. 2022Understanding and Improving Visual Prompting: A Label-Mapping Perspectivecited by 86
  32. 2021Backdoor Pre-trained Models Can Transfer to Allcited by 79
  33. 2023Diversity-Aware Meta Visual Promptingcited by 57
  34. 2019Zero-shot Text Classification With Generative Language Modelscited by 55
  35. 2018Dialogue Learning with Human Teaching and Feedback in End-to-End Trainable Task-Oriented Dialogue Systemscited by 53
  36. 2021Poisoning the Unlabeled Dataset of Semi-Supervised Learningcited by 44
  37. 2022Unleashing the Power of Visual Prompting At the Pixel Levelcited by 37
  38. 2022Visual Prompting for Adversarial Robustnesscited by 33

50 most cited of 56 All 56 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 2; 2026: 1.

4 papers.

  1. 2025Privacy-Preserving Personalized Federated Prompt Learning for Multimodal Large Language Modelscited by 5
  2. 2024Encapsulating Knowledge in One Promptcited by 3
  3. 2025Backdoor Attacks on Open Vocabulary Object Detectors via Multi-Modal Prompt Tuningcited by 0
  4. 2026BadBone: Backdoor Attacks Against Backbone Models in Visual Prompt Learningcited by 0