mlsanityData to 7 Oct 2026

Exploring the Universal Vulnerability of Prompt-based Learning Paradigm

arXiv 2204.05239 ↗First posted 11 Apr 2022cs.CLcited by 56

Not on the map. Why?

Cited by 28 in LLM safety and security, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Prompt-based learning paradigm bridges the gap between pre-training and fine-tuning, and works effectively under the few-shot setting. However, we find that this learning paradigm inherits the vulnerability from the pre-training stage, where model predictions can be misled by inserting certain triggers into the text. In this paper, we explore this universal vulnerability by either injecting backdoor triggers or searching for adversarial triggers on pre-trained language models using only plain text. In both scenarios, we demonstrate that our triggers can totally control or severely decrease the performance of prompt-based models fine-tuned on arbitrary downstream tasks, reflecting the universal vulnerability of the prompt-based learning paradigm. Further experiments show that adversarial triggers have good transferability among language models. We also find conventional fine-tuning models are not vulnerable to adversarial triggers constructed from pre-trained language models. We conclude by proposing a potential solution to mitigate our attack methods. Code and data are publicly available at https://github.com/leix28/prompt-universal-vulnerability

Key facts. Cited by 56 papers in the map (data through 7 Oct 2026). Builds on 24 of its 36 references that are in the map. First posted on arXiv 11 Apr 2022.

Builds on

24 of 36 references.

References · 24

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  4. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  5. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  6. 2016Pointer Sentinel Mixture Modelscited by 2,992
  7. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  8. 2019Language Models as Knowledge Bases?cited by 1,922
  9. 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,255
  10. 2020Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inferencecited by 934
  11. 2019Universal Adversarial Triggers for Attacking and Analyzing NLPcited by 709
  12. 2018TextBugger: Generating Adversarial Text Against Real-world Applicationscited by 376
12 more
  1. 2020Weight Poisoning Attacks on Pre-trained Modelscited by 318
  2. 2020AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Promptscited by 246
  3. 2020ONION: A Simple and Effective Defense Against Textual Backdoor Attackscited by 231
  4. 2019A backdoor attack against LSTM-based text classification systemscited by 214
  5. 2021Hidden Killer: Invisible Textual Backdoor Attacks with Syntactic Triggercited by 203
  6. 2021Mind the Style of Text! Adversarial and Backdoor Attacks Based on Text Style Transfercited by 147
  7. 2021Cutting Down on Prompts and Parameters: Simple Few-Shot Learning with Language Modelscited by 139
  8. 2021Hidden Backdoors in Human-Centric Language Modelscited by 83
  9. 2021Red Alarm for Pre-trained Models: Universal Vulnerability to Neuron-Level Backdoor Attackscited by 44
  10. 2020CAT-Gen: Improving Robustness in NLP Models via Controlled Adversarial Text Generationcited by 43
  11. 2021Multi-granularity Textual Adversarial Attack with Behavior Cloningcited by 21
  12. 2017Satirical News Detection and Analysis using Attention Mechanism and Linguistic Featurescited by 14

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 6; 2023: 22; 2024: 20; 2025: 7; 2026: 1.

56 papers.

Sort:

By area:

  1. 2022RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learningcited by 361
  2. 2023Survey of Vulnerabilities in Large Language Models Revealed by Adversarial Attackscited by 189
  3. 2024Security and Privacy Challenges of Large Language Models: A Surveycited by 179
  4. 2022Why Should Adversarial Perturbations be Imperceptible? Rethink the Research Paradigm in Adversarial NLPcited by 99
  5. 2023DRESS: Instructing Large Vision-Language Models to Align and Interact with Humans via Natural Language Feedbackcited by 99
  6. 2023Jatmo: Prompt Injection Defense by Task-Specific Finetuningcited by 89
  7. 2023Prompt as Triggers for Backdoor Attack: Examining the Vulnerability in Language Modelscited by 87
  8. 2023Use of LLMs for Illicit Purposes: Threats, Prevention Measures, and Vulnerabilitiescited by 76
  9. 2024BadChain: Backdoor Chain-of-Thought Prompting for Large Language Modelscited by 74
  10. 2022A Unified Evaluation of Textual Backdoor Learning: Frameworks and Benchmarkscited by 70
  11. 2023Black Box Adversarial Prompting for Foundation Modelscited by 62
  12. 2022BadPrompt: Backdoor Attacks on Continuous Promptscited by 59
38 more
  1. 2024Universal Vulnerabilities in Large Language Models: Backdoor Attacks for In-context Learningcited by 47
  2. 2023PoisonPrompt: Backdoor Attack on Prompt-based Large Language Modelscited by 40
  3. 2024Securing Large Language Models: Threats, Vulnerabilities and Responsible Practicescited by 36
  4. 2023NOTABLE: Transferable Backdoor Attacks Against Prompt-based NLP Modelscited by 33
  5. 2023Defending Pre-trained Language Models as Few-shot Learners against Backdoor Attackscited by 32
  6. 2024PoisonBench: Assessing Large Language Model Vulnerability to Data Poisoningcited by 31
  7. 2023TrojLLM: A Black-box Trojan Prompt Attack on Large Language Modelscited by 29
  8. 2024Universal Prompt Optimizer for Safe Text-to-Image Generationcited by 27
  9. 2023Evaluating the Robustness of Text-to-image Diffusion Models against Real-world Attackscited by 24
  10. 2023Attacks in Adversarial Machine Learning: A Systematic Survey from the Life-cycle Perspectivecited by 19
  11. 2023ParaFuzz: An Interpretability-Driven Technique for Detecting Poisoned Samples in NLPcited by 16
  12. 2024Defending Against Weight-Poisoning Backdoor Attacks for Parameter-Efficient Fine-Tuningcited by 16
  13. 2022PromptAttack: Prompt-based Attack for Language Models via Gradient Searchcited by 15
  14. 2024From Transparency to Accountability and Back: A Discussion of Access and Evidence in AI Auditingcited by 15
  15. 2024TuBA: Cross-Lingual Transferability of Backdoor Attacks in LLMs with Instruction Tuningcited by 13
  16. 2024Uncertainty is Fragile: Manipulating Uncertainty in Large Language Modelscited by 13
  17. 2023A Systematic Survey of Prompt Engineering on Vision-Language Foundation Modelscited by 12
  18. 2023Forcing Generative Models to Degenerate Ones: The Power of Data Poisoning Attackscited by 9
  19. 2024Turning Generative Models Degenerate: The Power of Data Poisoning Attackscited by 9
  20. 2024Stepwise Reasoning Error Disruption Attack of LLMscited by 8
  21. 2025A Systematic Review of Poisoning Attacks Against Large Language Modelscited by 8
  22. 2022QueryForm: A Simple Zero-shot Form Entity Query Frameworkcited by 7
  23. 2023Revisiting Automated Prompting: Are We Actually Doing Better?cited by 7
  24. 2023LMSanitator: Defending Prompt-Tuning Against Task-Agnostic Backdoorscited by 7
  25. 2023Developing Effective Educational Chatbots with ChatGPT prompts: Insights from Preliminary Tests in a Case Study on Social Media Literacy (with appendix)cited by 6
  26. 2023Learning to Prompt in the Classroom to Understand AI Limits: A pilot studycited by 6
  27. 2024Test-Time Multimodal Backdoor Detection by Contrastive Promptingcited by 5
  28. 2024Can LLMs be Fooled? Investigating Vulnerabilities in LLMscited by 5
  29. 2024Fine-tuned Large Language Models (LLMs): Improved Prompt Injection Attacks Detectioncited by 4
  30. 2023COVER: A Heuristic Greedy Adversarial Attack on Prompt-based Learning in Language Modelscited by 3
  31. 2023TARGET: Template-Transferable Backdoor Attack Against Prompt-based NLP Models via GPT4cited by 3
  32. 2025Automating Prompt Leakage Attacks on Large Language Models Using Agentic Approachcited by 3
  33. 2025Pay Attention to the Triggers: Constructing Backdoors That Survive Distillationcited by 3
  34. 2026Why LoRA Fails to Forget: Regularized Low-Rank Adaptation Against Backdoors in Language Modelscited by 3
  35. 2024Injecting Undetectable Backdoors in Obfuscated Neural Networks and Language Modelscited by 2
  36. 2024Breaking PEFT Limitations: Leveraging Weak-to-Strong Knowledge Transfer for Backdoor Attacks in LLMscited by 2
  37. 2024New Emerged Security and Privacy of Pre-trained Model: a Survey and Outlookcited by 2
  38. 2025Are My Optimized Prompts Compromised? Exploring Vulnerabilities of LLM-based Optimizerscited by 2

50 most cited of 56 All 56 (JSON)