mlsanityData to 14 Oct 2026

Prompt as Triggers for Backdoor Attack: Examining the Vulnerability in Language Models

arXiv 2305.01219 ↗First posted 2 May 2023cs.CL cs.AIcited by 94

On the map: LLM safety and security → Backdoor and data poisoning attacks

Abstract (arXiv)

The prompt-based learning paradigm, which bridges the gap between pre-training and fine-tuning, achieves state-of-the-art performance on several NLP tasks, particularly in few-shot settings. Despite being widely applied, prompt-based learning is vulnerable to backdoor attacks. Textual backdoor attacks are designed to introduce targeted vulnerabilities into models by poisoning a subset of training samples through trigger injection and label modification. However, they suffer from flaws such as abnormal natural language expressions resulting from the trigger and incorrect labeling of poisoned samples. In this study, we propose ProAttack, a novel and efficient method for performing clean-label backdoor attacks based on the prompt, which uses the prompt itself as a trigger. Our method does not require external triggers and ensures correct labeling of poisoned samples, improving the stealthy nature of the backdoor attack. With extensive experiments on rich-resource and few-shot text classification tasks, we empirically validate ProAttack's competitive performance in textual backdoor attacks. Notably, in the rich-resource setting, ProAttack achieves state-of-the-art attack success rates in the clean-label backdoor attack benchmark without external triggers.

Key facts. Cited by 94 papers in the map (data through 14 Oct 2026). Placed in Backdoor and data poisoning attacks, one of 12 directions in LLM safety and security. Builds on 42 of its 57 references that are in the map. First posted on arXiv 2 May 2023.

Builds on

42 of 57 references.

Closest ancestors · 6

Same direction, newest first

  1. 2023Backdoor Attacks with Input-unique Triggers in NLP25 Mar
  2. 2022BadPrompt: Backdoor Attacks on Continuous Prompts27 Nov
  3. 2022Kallima: A Clean-label Framework for Textual Backdoor Attacks3 Jun
  4. 2021Triggerless Backdoor Attack for NLP Tasks with Clean Labels15 Nov
  5. 2021Turn the Combination Lock: Learnable Textual Backdoor Attacks via Word Substitution11 Jun
  6. 2021Hidden Killer: Invisible Textual Backdoor Attacks with Syntactic Trigger26 May

Other references · 36

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2023GPT-4 Technical Reportcited by 18,396
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  5. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  6. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  7. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,899
  8. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,915
  9. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
  10. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,811
  11. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,411
  12. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,234
24 more
  1. 2019Language Models as Knowledge Bases?cited by 1,936
  2. 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,256
  3. 2017BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chaincited by 1,106
  4. 2020Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inferencecited by 938
  5. 2022Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learningcited by 896
  6. 2021GPT Understands, Toocited by 716
  7. 2018Spectral Signatures in Backdoor Attackscited by 484
  8. 2020Weight Poisoning Attacks on Pre-trained Modelscited by 341
  9. 2019Predicting the Type and Target of Offensive Posts in Social Mediacited by 271
  10. 2020ONION: A Simple and Effective Defense Against Textual Backdoor Attackscited by 263
  11. 2020Clean-Label Backdoor Attacks on Video Recognition Modelscited by 169
  12. 2021Noisy Channel Language Model Prompting for Few-Shot Text Classificationcited by 165
  13. 2021Backdoor Attacks on Pre-trained Models by Layerwise Weight Poisoningcited by 113
  14. 2020Mitigating backdoor attacks in LSTM-based Text Classification Systems by Backdoor Keyword Identificationcited by 86
  15. 2021Towards Robustness Against Natural Language Word Substitutionscited by 69
  16. 2022Exploring the Universal Vulnerability of Prompt-based Learning Paradigmcited by 57
  17. 2021How Should Pre-Trained Language Models Be Fine-Tuned Towards Adversarial Robustness?cited by 49
  18. 2021CINS: Comprehensive Instruction for Few-shot Learning in Task-oriented Dialog Systemscited by 30
  19. 2022BadHash: Invisible Backdoor Attacks against Deep Hashing with Clean Labelcited by 28
  20. 2021FewshotQA: A simple framework for few-shot learning of question answering tasks using pre-trained text-to-text modelscited by 26
  21. 2022Certified Robustness Against Natural Language Attacks by Causal Interventioncited by 24
  22. 2021Improving Neural Cross-Lingual Summarization via Employing Optimal Transport Distance for Knowledge Distillationcited by 22
  23. 2021The Power of Prompt Tuning for Low-Resource Semantic Parsingcited by 21
  24. 2022Textual Manifold-based Defense Against Natural Language Adversarial Examplescited by 19

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 16; 2024: 42; 2025: 24; 2026: 12.

94 papers.

Sort:

By area:

  1. 2023Challenges and Applications of Large Language Modelscited by 254
  2. 2023Large Language Model Alignment: A Surveycited by 243
  3. 2024Security and Privacy Challenges of Large Language Models: A Surveycited by 186
  4. 2023ReviewerGPT? An Exploratory Study on Using Large Language Models for Paper Reviewingcited by 122
  5. 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deploymentcited by 116
  6. 2024BadChain: Backdoor Chain-of-Thought Prompting for Large Language Modelscited by 80
  7. 2023Composite Backdoor Attacks Against Large Language Modelscited by 73
  8. 2023Unleashing the potential of prompt engineering for large language modelscited by 72
  9. 2024TrojanRAG: Retrieval-Augmented Generation Can Be Backdoor Driver in Large Language Modelscited by 67
  10. 2024Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systemscited by 66
  11. 2024Instruction Backdoor Attacks Against Customized LLMscited by 62
  12. 2024Universal Vulnerabilities in Large Language Models: Backdoor Attacks for In-context Learningcited by 53
38 more
  1. 2023Privacy in Large Language Models: Attacks, Defenses and Future Directionscited by 51
  2. 2023Tricking LLMs into Disobedience: Formalizing, Analyzing, and Detecting Jailbreakscited by 49
  3. 2024Breaking Down the Defenses: A Comparative Survey of Attacks on Large Language Modelscited by 44
  4. 2023A Comprehensive Overview of Backdoor Attacks in Large Language Models within Communication Networkscited by 42
  5. 2023Exploring the Potential of Large Language Models in Computational Argumentationcited by 42
  6. 2023Backdoor Attacks and Countermeasures in Natural Language Processing Models: A Comprehensive Security Reviewcited by 39
  7. 2024Securing Large Language Models: Threats, Vulnerabilities and Responsible Practicescited by 37
  8. 2024Feedback-Guided Extraction of Knowledge Base from Retrieval-Augmented LLM Applicationscited by 37
  9. 2024Advances and Open Challenges in Federated Foundation Modelscited by 36
  10. 2024A Survey of Recent Backdoor Attacks and Defenses in Large Language Modelscited by 36
  11. 2025MCIP: Protecting MCP Safety via Model Contextual Integrity Protocolcited by 34
  12. 2024PoisonBench: Assessing Large Language Model Vulnerability to Data Poisoningcited by 33
  13. 2024CROW: Eliminating Backdoors from Large Language Models via Internal Consistency Regularizationcited by 33
  14. 2023Identifying and Mitigating Privacy Risks Stemming from Language Models: A Surveycited by 30
  15. 2024Scaling Trends for Data Poisoning in LLMscited by 30
  16. 2024Unlearning Backdoor Attacks for LLMs with Weak-to-Strong Knowledge Distillationcited by 27
  17. 2024Data Poisoning for In-context Learningcited by 21
  18. 2024Token-wise Influential Training Data Retrieval for Large Language Modelscited by 20
  19. 2024Preemptive Answer "Attacks" on Chain-of-Thought Reasoningcited by 19
  20. 2024PROMPTFUZZ: Harnessing Fuzzing Techniques for Robust Testing of Prompt Injection in LLMscited by 17
  21. 2024Defending Against Weight-Poisoning Backdoor Attacks for Parameter-Efficient Fine-Tuningcited by 16
  22. 2023A White-Box False Positive Adversarial Attack Method on Contrastive Loss Based Offline Handwritten Signature Verification Modelscited by 15
  23. 2024Mitigating Backdoor Threats to Large Language Models: Advancement and Challengescited by 15
  24. 2024Backdoor Attacks on Dense Retrieval via Public and Unintentional Triggerscited by 14
  25. 2024TuBA: Cross-Lingual Transferability of Backdoor Attacks in LLMs with Instruction Tuningcited by 14
  26. 2025Uni-Retrieval: A Multi-Style Retrieval Framework for STEM's Educationcited by 13
  27. 2024Simulate and Eliminate: Revoke Backdoors for Generative Large Language Modelscited by 12
  28. 2023Causality Analysis for Evaluating the Security of Large Language Modelscited by 11
  29. 2024Exploring Vulnerabilities and Protections in Large Language Models: A Surveycited by 11
  30. 2024Artwork Protection Against Neural Style Transfer Using Locally Adaptive Adversarial Color Attackcited by 10
  31. 2023Forcing Generative Models to Degenerate Ones: The Power of Data Poisoning Attackscited by 9
  32. 2024Turning Generative Models Degenerate: The Power of Data Poisoning Attackscited by 9
  33. 2025Erased but Not Forgotten: How Backdoors Compromise Concept Erasurecited by 9
  34. 2025BEAT: Visual Backdoor Attacks on VLM-based Embodied Agents via Contrastive Trigger Learningcited by 9
  35. 2024Imperio: Language-Guided Backdoor Attacks for Arbitrary Model Controlcited by 8
  36. 2024A Survey on Responsible Generative AI: What to Generate and What Notcited by 8
  37. 2024Blockchain for Large Language Model Security and Safety: A Holistic Surveycited by 8
  38. 2024Recent advancements in LLM Red-Teaming: Techniques, Defenses, and Ethical Considerationscited by 8

50 most cited of 94 All 94 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2022BITE: Textual Backdoor Attacks with Iterative Trigger Injection
  2. 2022A Unified Evaluation of Textual Backdoor Learning: Frameworks and Benchmarks
  3. 2023PoisonPrompt: Backdoor Attack on Prompt-based Large Language Models
  4. 2023Large Language Models Are Better Adversaries: Exploring Generative Clean-Label Backdoor Attacks Against Text Classifiers
  5. 2021BadPre: Task-agnostic Backdoor Attacks to Pre-trained NLP Foundation Models
  6. 2023Mitigating Backdoor Poisoning Attacks through the Lens of Spurious Correlation
  7. 2025Strategic Sample Selection for Improved Clean-Label Backdoor Attacks in Text Classification
  8. 2023NOTABLE: Transferable Backdoor Attacks Against Prompt-based NLP Models