mlsanityData to 7 Oct 2026

A Survey of Recent Backdoor Attacks and Defenses in Large Language Models

arXiv 2406.06852 ↗First posted 10 Jun 2024cs.CR cs.AI cs.CLcited by 36

On the map: LLM safety and security → Backdoor and data poisoning attacks

Abstract (arXiv)

Large Language Models (LLMs), which bridge the gap between human language understanding and complex problem-solving, achieve state-of-the-art performance on several NLP tasks, particularly in few-shot and zero-shot settings. Despite the demonstrable efficacy of LLMs, due to constraints on computational resources, users have to engage with open-source language models or outsource the entire training process to third-party platforms. However, research has demonstrated that language models are susceptible to potential security vulnerabilities, particularly in backdoor attacks. Backdoor attacks are designed to introduce targeted vulnerabilities into language models by poisoning training samples or model weights, allowing attackers to manipulate model responses through malicious triggers. While existing surveys on backdoor attacks provide a comprehensive overview, they lack an in-depth examination of backdoor attacks specifically targeting LLMs. To bridge this gap and grasp the latest trends in the field, this paper presents a novel perspective on backdoor attacks for LLMs by focusing on fine-tuning methods. Specifically, we systematically classify backdoor attacks into three categories: full-parameter fine-tuning, parameter-efficient fine-tuning, and no fine-tuning Based on insights from a substantial review, we also discuss crucial issues for future research on backdoor attacks, such as further exploring attack algorithms that do not require fine-tuning, or developing more covert attack algorithms.

Key facts. Cited by 36 papers in the map (data through 7 Oct 2026). Placed in Backdoor and data poisoning attacks, one of 12 directions in LLM safety and security. Builds on 113 of its 177 references that are in the map. First posted on arXiv 10 Jun 2024.

Builds on

113 of 177 references.

Closest ancestors · 41

Same direction, newest first

  1. 2024Cross-Context Backdoor Attacks against Graph Prompt Learning28 May
  2. 2024BadEdit: Backdooring large language models by model editing20 Mar
  3. 2024Universal Vulnerabilities in Large Language Models: Backdoor Attacks for In-context Learning11 Jan
  4. 2024Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training10 Jan
  5. 2023Forcing Generative Models to Degenerate Ones: The Power of Data Poisoning Attacks7 Dec
  6. 2023TARGET: Template-Transferable Backdoor Attack Against Prompt-based NLP Models via GPT429 Nov
  7. 2023TextGuard: Provable Defense against Backdoor Attacks on Text Classification19 Nov
  8. 2023Test-time Backdoor Mitigation for Black-Box Large Language Models with Defensive Demonstrations16 Nov
  9. 2023Stealthy and Persistent Unalignment on Large Language Models via Backdoor Injections15 Nov
  10. 2023Large Language Models Are Better Adversaries: Exploring Generative Clean-Label Backdoor Attacks Against Text Classifiers28 Oct
  11. 2023PoisonPrompt: Backdoor Attack on Prompt-based Large Language Models19 Oct
  12. 2023Composite Backdoor Attacks Against Large Language Models11 Oct
29 more
  1. 2023Defending Pre-trained Language Models as Few-shot Learners against Backdoor Attacks23 Sep
  2. 2023Backdoor Attacks and Countermeasures in Natural Language Processing Models: A Comprehensive Security Review12 Sep
  3. 2023LMSanitator: Defending Prompt-Tuning Against Task-Agnostic Backdoors26 Aug
  4. 2023Backdooring Instruction-Tuned Large Language Models with Virtual Prompt Injection31 Jul
  5. 2023Backdoor Attacks for In-Context Learning with Language Models27 Jul
  6. 2023Adversarial Clean Label Backdoor Attacks and Defenses on Text Classification Systems31 May
  7. 2023Instructions as Backdoors: Backdoor Vulnerabilities of Instruction Tuning for Large Language Models24 May
  8. 2023UOR: Universal Backdoor Attacks on Pre-trained Language Models16 May
  9. 2023Defending against Insertion-based Textual Backdoor Attacks via Attribution3 May
  10. 2023Prompt as Triggers for Backdoor Attack: Examining the Vulnerability in Language Models2 May
  11. 2023ChatGPT as an Attack Tool: Stealthy Textual Backdoor Attack via Blackbox Generative Model Trigger27 Apr
  12. 2023Backdoor Attacks with Input-unique Triggers in NLP25 Mar
  13. 2023NCL: Textual Backdoor Defense Using Noise-augmented Contrastive Learning3 Mar
  14. 2023BadGPT: Exploring Security Vulnerabilities of ChatGPT via Backdoor Attacks to InstructGPT21 Feb
  15. 2023Training-free Lexical Backdoor Attacks on Language Models8 Feb
  16. 2023BDMMT: Backdoor Sample Detection for Language Models through Model Mutation Testing25 Jan
  17. 2022Fine-mixing: Mitigating Backdoors in Fine-tuned Language Models18 Oct
  18. 2022WeDef: Weakly Supervised Backdoor Defense for Text Classification24 May
  19. 2022Constrained Optimization with Dynamic Bound-scaling for Effective NLPBackdoor Defense11 Feb
  20. 2021Spinning Language Models: Risks of Propaganda-As-A-Service and Countermeasures9 Dec
  21. 2021Triggerless Backdoor Attack for NLP Tasks with Clean Labels15 Nov
  22. 2021RAP: Robustness-Aware Perturbations for Defending against Backdoor Attacks on NLP Models15 Oct
  23. 2021BadPre: Task-agnostic Backdoor Attacks to Pre-trained NLP Foundation Models6 Oct
  24. 2021BFClass: A Backdoor-free Text Classification Framework22 Sep
  25. 2021Defending Against Backdoor Attacks in Natural Language Generation3 Jun
  26. 2021Hidden Killer: Invisible Textual Backdoor Attacks with Syntactic Trigger26 May
  27. 2021Hidden Backdoors in Human-Centric Language Models1 May
  28. 2021Be Careful about Poisoned Word Embeddings: Exploring the Vulnerability of the Embedding Layers in NLP Models29 Mar
  29. 2021T-Miner: A Generative Approach to Defend Against Trojan Attacks on DNN-based Text Classification7 Mar

Other references · 72

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  5. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  6. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  7. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  8. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  9. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  10. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  11. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  12. 2017Prototypical Networks for Few-shot Learningcited by 3,001
38 more
  1. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
  2. 2015Character-level Convolutional Networks for Text Classificationcited by 2,293
  3. 2018Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarizationcited by 1,308
  4. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,106
  5. 2015Teaching Machines to Read and Comprehendcited by 1,039
  6. 2023An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuningcited by 622
  7. 2018Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networkscited by 550
  8. 2017Zero-Shot Learning -- A Comprehensive Evaluation of the Good, the Bad and the Uglycited by 536
  9. 2023DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Modelscited by 505
  10. 2023SmoothLLM: Defending Large Language Models Against Jailbreaking Attackscited by 434
  11. 2020Weight Poisoning Attacks on Pre-trained Modelscited by 318
  12. 2021Neural Attention Distillation: Erasing Backdoor Triggers from Deep Neural Networkscited by 313
  13. 2018Newsroom: A Dataset of 1.3 Million Summaries with Diverse Extractive Strategiescited by 292
  14. 2019Predicting the Type and Target of Offensive Posts in Social Mediacited by 268
  15. 2023Poisoning Language Models During Instruction Tuningcited by 254
  16. 2023Prompting Large Language Model for Machine Translation: A Case Studycited by 247
  17. 2024Rethinking Machine Unlearning for Large Language Modelscited by 247
  18. 2020ONION: A Simple and Effective Defense Against Textual Backdoor Attackscited by 231
  19. 2019A backdoor attack against LSTM-based text classification systemscited by 214
  20. 2018Hate Speech Dataset from a White Supremacy Forumcited by 196
  21. 2020BadNL: Backdoor Attacks against NLP Models with Semantic-preserving Improvementscited by 191
  22. 2023Defending Against Alignment-Breaking Attacks via Robustly Aligned LLMcited by 173
  23. 2023Fact-Checking Complex Claims with Program-Guided Reasoningcited by 146
  24. 2024Watch Out for Your Agents! Investigating Backdoor Threats to LLM-Based Agentscited by 135
  25. 2024Jailbreaking Attack against Multimodal Large Language Modelcited by 128
  26. 2021SPECTRE: Defending Against Backdoor Attacks Using Robust Statisticscited by 123
  27. 2019Demon in the Variant: Statistical Analysis of DNNs for Robust Backdoor Contamination Detectioncited by 121
  28. 2024Vaccine: Perturbation-aware Alignment for Large Language Models against Harmful Fine-tuning Attackcited by 113
  29. 2021Backdoor Attacks on Pre-trained Models by Layerwise Weight Poisoningcited by 103
  30. 2023Is RLHF More Difficult than Standard RL?cited by 96
  31. 2023The unreasonable effectiveness of few-shot learning for machine translationcited by 91
  32. 2020Trojaning Language Models for Fun and Profitcited by 80
  33. 2021Backdoor Pre-trained Models Can Transfer to Allcited by 79
  34. 2022Backdoor Defense with Machine Unlearningcited by 61
  35. 2023Are You Copying My Model? Protecting the Copyright of Large Language Models for EaaS via Backdoor Watermarkcited by 61
  36. 2020Can Adversarial Weight Perturbations Inject Neural Backdoors?cited by 51
  37. 2021How Should Pre-Trained Language Models Be Fine-Tuned Towards Adversarial Robustness?cited by 49
  38. 2022A Study of the Attention Abnormality in Trojaned BERTscited by 47

50 most cited of 72 All 72 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 16; 2026: 17.

36 papers.

Sort:

By area:

  1. 2025Safety at Scale: A Comprehensive Survey of Large Model and Agent Safetycited by 63
  2. 2024Instant Adversarial Purification with Adversarial Consistency Distillationcited by 14
  3. 2024A Gray-box Attack against Latent Diffusion Model-based Image Editing by Posterior Collapsecited by 13
  4. 2025A Survey of Attacks on Large Language Modelscited by 13
  5. 2025Byzantine-Robust Decentralized Coordination of LLM Agentscited by 13
  6. 2026The Trigger in the Haystack: Extracting and Reconstructing LLM Backdoor Triggerscited by 13
  7. 2024Quantized Delta Weight Is Safety Keepercited by 12
  8. 2025Scalable Fingerprinting of Large Language Modelscited by 11
  9. 2025Position: LLM Watermarking Should Align Stakeholders' Incentives for Practical Adoptioncited by 7
  10. 2025When Style Breaks Safety: Defending LLMs Against Superficial Style Alignmentcited by 6
  11. 2025Jailbreak-Tuning: Models Efficiently Learn Jailbreak Susceptibilitycited by 6
  12. 2026Purifying Generative LLMs from Backdoors without Prior Knowledge or Clean Referencecited by 6
24 more
  1. 2025Beyond Vulnerabilities: A Survey of Adversarial Attacks as Both Threats and Defenses in Computer Vision Systemscited by 4
  2. 2025From Text to Talk: Audio-Language Model Needs Non-Autoregressive Joint Trainingcited by 3
  3. 2025Towards more transferable adversarial attack in black-box mannercited by 2
  4. 2025Provably Safe Model Updatescited by 2
  5. 2025DISTIL: Data-Free Inversion of Suspicious Trojan Inputs via Latent Diffusioncited by 1
  6. 2026Leveraging Soft Prompts for Privacy Attacks in Federated Prompt Tuningcited by 1
  7. 2026Backdooring Bias in Large Language Modelscited by 1
  8. 2026Stealthy Backdoor Attacks against LLMs Based on Natural Style Triggerscited by 1
  9. 2026MetaBackdoor: Exploiting Positional Encoding as a Backdoor Attack Surface in LLMscited by 1
  10. 2025Data Security in Large Language Models: Risks, Defense, and Directionscited by 0
  11. 2025Pruning Cannot Hurt Robustness: Certified Trade-offs in Reinforcement Learningcited by 0
  12. 2025SAID: Safety-Aware Intent Defense via Prefix Probing for Large Language Modelscited by 0
  13. 2025Password-Activated Shutdown Protocols for Misaligned Frontier Agentscited by 0
  14. 2026Towards a Data-Parameter Correspondence for LLMs: A Preliminary Discussioncited by 0
  15. 2026Secret Stealing Attacks on Local LLM Fine-Tuning through Supply-Chain Model Code Backdoorscited by 0
  16. 2026On the Privacy of LLMs: An Ablation Studycited by 0
  17. 2026When Emotion Becomes Trigger: Emotion-style dynamic Backdoor Attack Parasitising Large Language Modelscited by 0
  18. 2026Fuzzing Large Language Models to Elicit Hidden Behaviourscited by 0
  19. 2026CSO-LLM: Class Subspace Orthogonalization for Post-Training Backdoor Detection and Trigger Inversion in LLMscited by 0
  20. 2026Early Detection of Distributed Backdoors in Multi-Agent LLM Systems: A Characterization Studycited by 0
  21. 2026Genotypic Triggers: Exposing Pharmacogenomic Blind Spots via Host-Specific Backdoors in Generative Antimicrobial Peptide Modelscited by 0
  22. 2026SpecGuard: Inference-Time Backdoor Detection For Freecited by 0
  23. 2026Backdoor Containment via Expert Quarantine and Shutdown in LLMscited by 0
  24. 2026Backdoor Purification for LoRA-Tuned LLMs via Null-Space Projectioncited by 0