mlsanityData to 7 Oct 2026

NOTABLE: Transferable Backdoor Attacks Against Prompt-based NLP Models

arXiv 2305.17826 ↗First posted 28 May 2023cs.CL cs.CRcited by 33

On the map: LLM safety and security → Backdoor and data poisoning attacks

Abstract (arXiv)

Prompt-based learning is vulnerable to backdoor attacks. Existing backdoor attacks against prompt-based models consider injecting backdoors into the entire embedding layers or word embedding vectors. Such attacks can be easily affected by retraining on downstream tasks and with different prompting strategies, limiting the transferability of backdoor attacks. In this work, we propose transferable backdoor attacks against prompt-based models, called NOTABLE, which is independent of downstream tasks and prompting strategies. Specifically, NOTABLE injects backdoors into the encoders of PLMs by utilizing an adaptive verbalizer to bind triggers to specific words (i.e., anchors). It activates the backdoor by pasting input with triggers to reach adversary-desired anchors, achieving independence from downstream tasks and prompting strategies. We conduct experiments on six NLP tasks, three popular models, and three prompting strategies. Empirical results show that NOTABLE achieves superior attack performance (i.e., attack success rate over 90% on all the datasets), and outperforms two state-of-the-art baselines. Evaluations on three defenses show the robustness of NOTABLE. Our code can be found at https://github.com/RU-System-Software-and-Security/Notable.

Key facts. Cited by 33 papers in the map (data through 7 Oct 2026). Placed in Backdoor and data poisoning attacks, one of 12 directions in LLM safety and security. Builds on 50 of its 65 references that are in the map. First posted on arXiv 28 May 2023.

Builds on

50 of 65 references.

Closest ancestors · 12

Same direction, newest first

  1. 2022BadPrompt: Backdoor Attacks on Continuous Prompts27 Nov
  2. 2022Rethinking the Reverse-engineering of Trojan Triggers27 Oct
  3. 2022Constrained Optimization with Dynamic Bound-scaling for Effective NLPBackdoor Defense11 Feb
  4. 2021Spinning Language Models: Risks of Propaganda-As-A-Service and Countermeasures9 Dec
  5. 2021Trigger Hunting with a Topological Prior for Trojan Detection15 Oct
  6. 2021RAP: Robustness-Aware Perturbations for Defending against Backdoor Attacks on NLP Models15 Oct
  7. 2021Mind the Style of Text! Adversarial and Backdoor Attacks Based on Text Style Transfer14 Oct
  8. 2021BadPre: Task-agnostic Backdoor Attacks to Pre-trained NLP Foundation Models6 Oct
  9. 2021BadEncoder: Backdoor Attacks to Pre-trained Encoders in Self-Supervised Learning1 Aug
  10. 2021Turn the Combination Lock: Learnable Textual Backdoor Attacks via Word Substitution11 Jun
  11. 2021Hidden Killer: Invisible Textual Backdoor Attacks with Syntactic Trigger26 May
  12. 2021T-Miner: A Generative Approach to Defend Against Trojan Attacks on DNN-based Text Classification7 Mar

Other references · 38

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  5. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  6. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  7. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  8. 2015Character-level Convolutional Networks for Text Classificationcited by 2,293
  9. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,221
  10. 2020Extracting Training Data from Large Language Modelscited by 2,144
  11. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,140
  12. 2019Language Models as Knowledge Bases?cited by 1,922
26 more
  1. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,538
  2. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,486
  3. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,462
  4. 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,255
  5. 2017BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chaincited by 1,052
  6. 2019How Can We Know What Language Models Know?cited by 953
  7. 2020Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inferencecited by 934
  8. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 847
  9. 2021P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Taskscited by 686
  10. 2019Adversarial Training for Free!cited by 638
  11. 2020Weight Poisoning Attacks on Pre-trained Modelscited by 318
  12. 2019Label-Consistent Backdoor Attackscited by 285
  13. 2019FreeLB: Enhanced Adversarial Training for Natural Language Understandingcited by 261
  14. 2020ONION: A Simple and Effective Defense Against Textual Backdoor Attackscited by 231
  15. 2020BadNL: Backdoor Attacks against NLP Models with Semantic-preserving Improvementscited by 191
  16. 2021Poisoning and Backdooring Contrastive Learningcited by 149
  17. 2021Backdoor Scanning for Deep Neural Networks through K-Arm Optimizationcited by 96
  18. 2022BppAttack: Stealthy and Efficient Trojan Attacks against Deep Neural Networks via Image Quantization and Contrastive Adversarial Learningcited by 82
  19. 2021Backdoor Pre-trained Models Can Transfer to Allcited by 79
  20. 2022Exploring the Universal Vulnerability of Prompt-based Learning Paradigmcited by 56
  21. 2021Red Alarm for Pre-trained Models: Universal Vulnerability to Neuron-Level Backdoor Attackscited by 44
  22. 2023UNICORN: A Unified Backdoor Trigger Inversion Frameworkcited by 44
  23. 2022Training with More Confidence: Mitigating Injected and Natural Backdoors During Trainingcited by 38
  24. 2021Topological Detection of Trojaned Neural Networkscited by 34
  25. 2023BackdoorBox: A Python Toolbox for Backdoor Learningcited by 25
  26. 2022Backdoor Vulnerabilities in Normally Trained Deep Learning Modelscited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 10; 2024: 18; 2025: 4; 2026: 1.

33 papers.

Sort:

By area:

  1. 2023Prompt Injection attack against LLM-integrated Applicationscited by 568
  2. 2023GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Promptscited by 520
  3. 2024BadChain: Backdoor Chain-of-Thought Prompting for Large Language Modelscited by 74
  4. 2023Privacy in Large Language Models: Attacks, Defenses and Future Directionscited by 48
  5. 2023PoisonPrompt: Backdoor Attack on Prompt-based Large Language Modelscited by 40
  6. 2024Securing Large Language Models: Threats, Vulnerabilities and Responsible Practicescited by 36
  7. 2024Feedback-Guided Extraction of Knowledge Base from Retrieval-Augmented LLM Applicationscited by 36
  8. 2024Advances and Open Challenges in Federated Foundation Modelscited by 33
  9. 2023Integration of Large Language Models and Federated Learningcited by 28
  10. 2023Grounding Foundation Models through Federated Transfer Learning: A General Frameworkcited by 24
  11. 2024Can We Trust Embodied Agents? Exploring Backdoor Attacks against Embodied LLM-based Decision-Making Systemscited by 23
  12. 2024BadActs: A Universal Backdoor Defense in the Activation Spacecited by 19
21 more
  1. 2024SeqMIA: Sequential-Metric Based Membership Inference Attackcited by 19
  2. 2024Preemptive Answer "Attacks" on Chain-of-Thought Reasoningcited by 16
  3. 2024Cross-Context Backdoor Attacks against Graph Prompt Learningcited by 14
  4. 2024Uncertainty is Fragile: Manipulating Uncertainty in Large Language Modelscited by 13
  5. 2024Dialectical Alignment: Resolving the Tension of 3H and Security Threats of LLMscited by 11
  6. 2024The Dark Side of Human Feedback: Poisoning Large Language Models via User Inputscited by 10
  7. 2023TrojFSP: Trojan Insertion in Few-shot Prompt Tuningcited by 9
  8. 2023Beyond Boundaries: A Comprehensive Survey of Transferable Attacks on AI Systemscited by 6
  9. 2024Invisible Backdoor Attack against Self-supervised Learningcited by 5
  10. 2024Can LLMs be Fooled? Investigating Vulnerabilities in LLMscited by 5
  11. 2024MEGen: Generative Backdoor into Large Language Models via Model Editingcited by 5
  12. 2024Data-centric NLP Backdoor Defense from the Lens of Memorizationcited by 5
  13. 2023Alteration-free and Model-agnostic Origin Attribution of Generated Imagescited by 4
  14. 2025Chain-of-Trigger: An Agentic Backdoor that Paradoxically Enhances Agentic Robustnesscited by 4
  15. 2024Injecting Undetectable Backdoors in Obfuscated Neural Networks and Language Modelscited by 2
  16. 2025NLP Security and Ethics, in the Wildcited by 2
  17. 2023PETA: Parameter-Efficient Trojan Attackscited by 1
  18. 2025Rounding-Guided Backdoor Injection in Deep Learning Model Quantizationcited by 1
  19. 2025Casting a SPELL: Sentence Pairing Exploration for LLM Limitation-breakingcited by 1
  20. 2026MetaBackdoor: Exploiting Positional Encoding as a Backdoor Attack Surface in LLMscited by 1
  21. 2024Shortcuts Arising from Contrast: Effective and Covert Clean-Label Attacks in Prompt-Based Learningcited by 0