mlsanityData to 7 Oct 2026

Defending Pre-trained Language Models as Few-shot Learners against Backdoor Attacks

arXiv 2309.13256 ↗First posted 23 Sep 2023cs.LG cs.AIcited by 32

On the map: LLM safety and security → Backdoor and data poisoning attacks

Abstract (arXiv)

Pre-trained language models (PLMs) have demonstrated remarkable performance as few-shot learners. However, their security risks under such settings are largely unexplored. In this work, we conduct a pilot study showing that PLMs as few-shot learners are highly vulnerable to backdoor attacks while existing defenses are inadequate due to the unique challenges of few-shot scenarios. To address such challenges, we advocate MDP, a novel lightweight, pluggable, and effective defense for PLMs as few-shot learners. Specifically, MDP leverages the gap between the masking-sensitivity of poisoned and clean samples: with reference to the limited few-shot data as distributional anchors, it compares the representations of given samples under varying masking and identifies poisoned samples as ones with significant variations. We show analytically that MDP creates an interesting dilemma for the attacker to choose between attack effectiveness and detection evasiveness. The empirical evaluation using benchmark datasets and representative attacks validates the efficacy of MDP.

Key facts. Cited by 32 papers in the map (data through 7 Oct 2026). Placed in Backdoor and data poisoning attacks, one of 12 directions in LLM safety and security. Builds on 35 of its 44 references that are in the map. First posted on arXiv 23 Sep 2023.

Builds on

35 of 44 references.

Closest ancestors · 5

Same direction, newest first

  1. 2022BadPrompt: Backdoor Attacks on Continuous Prompts27 Nov
  2. 2021RAP: Robustness-Aware Perturbations for Defending against Backdoor Attacks on NLP Models15 Oct
  3. 2021Mind the Style of Text! Adversarial and Backdoor Attacks Based on Text Style Transfer14 Oct
  4. 2021Hidden Killer: Invisible Textual Backdoor Attacks with Syntactic Trigger26 May
  5. 2021Be Careful about Poisoned Word Embeddings: Exploring the Vulnerability of the Embedding Layers in NLP Models29 Mar

Other references · 30

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  5. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  6. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  7. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  8. 2016Matching Networks for One Shot Learningcited by 2,246
  9. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  10. 2019Language Models as Knowledge Bases?cited by 1,922
  11. 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,255
  12. 2019Unsupervised Data Augmentation for Consistency Trainingcited by 1,057
18 more
  1. 2017BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chaincited by 1,052
  2. 2017Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoningcited by 1,012
  3. 2021GPT Understands, Toocited by 715
  4. 2020It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learnerscited by 668
  5. 2020Weight Poisoning Attacks on Pre-trained Modelscited by 318
  6. 2021Factual Probing Is [MASK]: Learning vs. Learning to Recallcited by 291
  7. 2016Adversarial Training Methods for Semi-Supervised Text Classificationcited by 280
  8. 2020ONION: A Simple and Effective Defense Against Textual Backdoor Attackscited by 231
  9. 2019A backdoor attack against LSTM-based text classification systemscited by 214
  10. 2021Differentiable Prompt Makes Pre-trained Language Models Better Few-shot Learnerscited by 115
  11. 2021Backdoor Attacks on Pre-trained Models by Layerwise Weight Poisoningcited by 103
  12. 2020Mitigating backdoor attacks in LSTM-based Text Classification Systems by Backdoor Keyword Identificationcited by 80
  13. 2020Trojaning Language Models for Fun and Profitcited by 80
  14. 2019Design and Evaluation of a Multi-Domain Trojan Detection Method on Deep Neural Networkscited by 66
  15. 2018Diverse Few-Shot Text Classification with Multiple Metricscited by 61
  16. 2022Exploring the Universal Vulnerability of Prompt-based Learning Paradigmcited by 56
  17. 2020Universal Natural Language Processing with Limited Annotations: Try Few-shot Textual Entailment as a Startcited by 44
  18. 2020Bridging Text and Knowledge with Multi-Prototype Embedding for Few-Shot Relational Triple Extractioncited by 18

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 14; 2025: 10; 2026: 7.

32 papers.

Sort:

By area:

  1. 2024Security and Privacy Challenges of Large Language Models: A Surveycited by 179
  2. 2024Know Your Limits: A Survey of Abstention in Large Language Modelscited by 116
  3. 2024A Survey of Recent Backdoor Attacks and Defenses in Large Language Modelscited by 36
  4. 2024RevPRAG: Revealing Poisoning Attacks in Retrieval-Augmented Generation through LLM Activation Analysiscited by 33
  5. 2024CROW: Eliminating Backdoors from Large Language Models via Internal Consistency Regularizationcited by 29
  6. 2025PR-Attack: Coordinated Prompt-RAG Attacks on Retrieval-Augmented Generation in Large Language Models via Bilevel Optimizationcited by 29
  7. 2023Grounding Foundation Models through Federated Transfer Learning: A General Frameworkcited by 24
  8. 2024CleanGen: Mitigating Backdoor Attacks for Generation Tasks in Large Language Modelscited by 20
  9. 2024BadActs: A Universal Backdoor Defense in the Activation Spacecited by 19
  10. 2024Chain-of-Scrutiny: Detecting Backdoor Attacks for Large Language Modelscited by 17
  11. 2025Probe before You Talk: Towards Black-box Defense against Backdoor Unalignment for Large Language Modelscited by 15
  12. 2024Exploring Vulnerabilities and Protections in Large Language Models: A Surveycited by 10
20 more
  1. 2024TUNI: A Textual Unimodal Detector for Identity Inference in CLIP Modelscited by 8
  2. 2024Obliviate: Neutralizing Task-agnostic Backdoors within the Parameter-efficient Fine-tuning Paradigmcited by 7
  3. 2026Purifying Generative LLMs from Backdoors without Prior Knowledge or Clean Referencecited by 6
  4. 2024TrojFM: Resource-efficient Backdoor Attacks against Very Large Foundation Modelscited by 4
  5. 2024Watch the Watcher! Backdoor Attacks on Security-Enhancing Diffusion Modelscited by 4
  6. 2025Uncovering Vulnerabilities of LLM-Assisted Cyber Threat Intelligencecited by 3
  7. 2024New Emerged Security and Privacy of Pre-trained Model: a Survey and Outlookcited by 2
  8. 2025The Ultimate Cookbook for Invisible Poison: Crafting Subtle Clean-Label Text Backdoors with Style Attributescited by 2
  9. 2026Merging Triggers, Breaking Backdoors: Defensive Poisoning for Instruction-Tuned Language Modelscited by 2
  10. 2026SafeGPT: Preventing Data Leakage and Unethical Outputs in Enterprise LLM Usecited by 2
  11. 2025LLM in the Middle: A Systematic Review of Threats and Mitigations to Real-World LLM-based Systemscited by 1
  12. 2025SCOUT: A Defense Against Data Poisoning Attacks in Fine-Tuned Language Modelscited by 1
  13. 2026PIDP-Attack: Combining Prompt Injection with Database Poisoning Attacks on Retrieval-Augmented Generation Systemscited by 1
  14. 2025TrojanDec: Data-free Detection of Trojan Inputs in Self-supervised Learningcited by 0
  15. 2025MuTRAP: Multi-trigger Trojans Attacking Robot Task Planning Systemscited by 0
  16. 2025Large Language Models in Cybersecurity: Applications, Vulnerabilities, and Defense Techniquescited by 0
  17. 2025Backdoor Samples Detection Based on Perturbation Discrepancy Consistency in Pre-trained Language Modelscited by 0
  18. 2026Membership Inference for Contrastive Pre-training Models with Text-only PII Queriescited by 0
  19. 2026Backdoor Containment via Expert Quarantine and Shutdown in LLMscited by 0
  20. 2026Backdoor Purification for LoRA-Tuned LLMs via Null-Space Projectioncited by 0