mlsanityData to 14 Oct 2026

Mitigating Backdoor Threats to Large Language Models: Advancement and Challenges

arXiv 2409.19993 ↗First posted 30 Sep 2024cs.CR cs.AI cs.CLcited by 15

On the map: LLM safety and security → Backdoor and data poisoning attacks

Abstract (arXiv)

The advancement of Large Language Models (LLMs) has significantly impacted various domains, including Web search, healthcare, and software development. However, as these models scale, they become more vulnerable to cybersecurity risks, particularly backdoor attacks. By exploiting the potent memorization capacity of LLMs, adversaries can easily inject backdoors into LLMs by manipulating a small portion of training data, leading to malicious behaviors in downstream applications whenever the hidden backdoor is activated by the pre-defined triggers. Moreover, emerging learning paradigms like instruction tuning and reinforcement learning from human feedback (RLHF) exacerbate these risks as they rely heavily on crowdsourced data and human feedback, which are not fully controlled. In this paper, we present a comprehensive survey of emerging backdoor threats to LLMs that appear during LLM development or inference, and cover recent advancement in both defense and detection strategies for mitigating backdoor threats to LLMs. We also outline key challenges in addressing these threats, highlighting areas for future research.

Key facts. Cited by 15 papers in the map (data through 14 Oct 2026). Placed in Backdoor and data poisoning attacks, one of 12 directions in LLM safety and security. Builds on 98 of its 122 references that are in the map. First posted on arXiv 30 Sep 2024.

Builds on

98 of 122 references.

Closest ancestors · 29

Same direction, newest first

  1. 2024Transferring Backdoors between Large Language Models by Knowledge Distillation19 Aug
  2. 2024Uncertainty is Fragile: Manipulating Uncertainty in Large Language Models15 Jul
  3. 2024Future Events as Backdoor Triggers: Investigating Temporal Vulnerabilities in LLMs4 Jul
  4. 2024TrojanRAG: Retrieval-Augmented Generation Can Be Backdoor Driver in Large Language Models22 May
  5. 2024Exploring Backdoor Vulnerabilities of Chat Models3 Apr
  6. 2024BadEdit: Backdooring large language models by model editing20 Mar
  7. 2024BadChain: Backdoor Chain-of-Thought Prompting for Large Language Models20 Jan
  8. 2024Universal Vulnerabilities in Large Language Models: Backdoor Attacks for In-context Learning11 Jan
  9. 2024Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training10 Jan
  10. 2023Test-time Backdoor Mitigation for Black-Box Large Language Models with Defensive Demonstrations16 Nov
  11. 2023Backdooring Instruction-Tuned Large Language Models with Virtual Prompt Injection31 Jul
  12. 2023Backdoor Attacks for In-Context Learning with Language Models27 Jul
17 more
  1. 2023On the Exploitability of Instruction Tuning28 Jun
  2. 2023Multi-target Backdoor Attacks for Code Pre-trained Models14 Jun
  3. 2023Instructions as Backdoors: Backdoor Vulnerabilities of Instruction Tuning for Large Language Models24 May
  4. 2023Defending against Insertion-based Textual Backdoor Attacks via Attribution3 May
  5. 2023Prompt as Triggers for Backdoor Attack: Examining the Vulnerability in Language Models2 May
  6. 2023ChatGPT as an Attack Tool: Stealthy Textual Backdoor Attack via Blackbox Generative Model Trigger27 Apr
  7. 2023BadGPT: Exploring Security Vulnerabilities of ChatGPT via Backdoor Attacks to InstructGPT21 Feb
  8. 2023Backdoor Attacks to Pre-trained Unified Foundation Models18 Feb
  9. 2023Stealthy Backdoor Attack for Code Models6 Jan
  10. 2022Fine-mixing: Mitigating Backdoors in Fine-tuned Language Models18 Oct
  11. 2022BITE: Textual Backdoor Attacks with Iterative Trigger Injection25 May
  12. 2021Textual Backdoor Attacks Can Be More Harmful via Two Simple Tricks15 Oct
  13. 2021RAP: Robustness-Aware Perturbations for Defending against Backdoor Attacks on NLP Models15 Oct
  14. 2021Mind the Style of Text! Adversarial and Backdoor Attacks Based on Text Style Transfer14 Oct
  15. 2021Trojan Signatures in DNN Weights7 Sep
  16. 2021Hidden Killer: Invisible Textual Backdoor Attacks with Syntactic Trigger26 May
  17. 2021T-Miner: A Generative Approach to Defend Against Trojan Attacks on DNN-based Text Classification7 Mar

Other references · 69

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2023GPT-4 Technical Reportcited by 18,396
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  5. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  6. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  7. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  8. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  9. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  10. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
  11. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,242
  12. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,470
38 more
  1. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,439
  2. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,154
  3. 2017BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chaincited by 1,106
  4. 2023Large Language Models as Optimizerscited by 864
  5. 2023Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedbackcited by 759
  6. 2019Universal Adversarial Triggers for Attacking and Analyzing NLPcited by 722
  7. 2021Cross-Task Generalization via Natural Language Crowdsourcing Instructionscited by 699
  8. 2018Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networkscited by 580
  9. 2022Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Modelcited by 559
  10. 2022BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Miningcited by 538
  11. 2019STRIP: A Defence Against Trojan Attacks on Deep Neural Networkscited by 459
  12. 2019FinBERT: Financial Sentiment Analysis with Pre-trained Language Modelscited by 385
  13. 2023GPT-Driver: Learning to Drive with GPTcited by 347
  14. 2023Poisoning Web-Scale Training Datasets is Practicalcited by 290
  15. 2024TrustLLM: Trustworthiness in Large Language Modelscited by 285
  16. 2023Poisoning Language Models During Instruction Tuningcited by 269
  17. 2020ONION: A Simple and Effective Defense Against Textual Backdoor Attackscited by 263
  18. 2019A backdoor attack against LSTM-based text classification systemscited by 227
  19. 2019Detecting AI Trojans Using Meta Neural Analysiscited by 199
  20. 2019Certified Robustness to Adversarial Word Substitutionscited by 168
  21. 2021Adversarial Unlearning of Backdoors via Implicit Hypergradientcited by 166
  22. 2017Neural Trojanscited by 153
  23. 2024Optimization-based Prompt Injection Attack to LLM-as-a-Judgecited by 143
  24. 2024Watch Out for Your Agents! Investigating Backdoor Threats to LLM-Based Agentscited by 137
  25. 2022PromptChainer: Chaining Large Language Model Prompts through Visual Programmingcited by 134
  26. 2023Universal Jailbreak Backdoors from Poisoned Human Feedbackcited by 128
  27. 2024BadAgent: Inserting and Activating Backdoor Attacks in LLM Agentscited by 115
  28. 2019Robust Anomaly Detection and Backdoor Attack Detection Via Differential Privacycited by 95
  29. 2020Mitigating backdoor attacks in LSTM-based Text Classification Systems by Backdoor Keyword Identificationcited by 86
  30. 2024BadRAG: Identifying Vulnerabilities in Retrieval Augmented Generation of Large Language Modelscited by 86
  31. 2021Backdoor Attack in the Physical Worldcited by 84
  32. 2023Cognitive Overload: Jailbreaking Large Language Models with Overloaded Logical Thinkingcited by 81
  33. 2021Black-box Detection of Backdoor Attacks with Limited Information and Datacited by 73
  34. 2019Design and Evaluation of a Multi-Domain Trojan Detection Method on Deep Neural Networkscited by 71
  35. 2024Instructional Fingerprinting of Large Language Modelscited by 68
  36. 2024RigorLLM: Resilient Guardrails for Large Language Models against Undesired Contentcited by 68
  37. 2021Fine-Tuning Large Neural Language Models for Biomedical Natural Language Processingcited by 57
  38. 2022A Study of the Attention Abnormality in Trojaned BERTscited by 48

50 most cited of 69 All 69 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 8; 2026: 6.

15 papers.

Sort:

By area:

  1. 2025Backdoor Attribution: Elucidating and Controlling Backdoor in Language Modelscited by 10
  2. 2024SudoLM: Learning Access Control of Parametric Knowledge with Authorization Alignmentcited by 8
  3. 2025P2P: A Poison-to-Poison Remedy for Reliable Backdoor Defense in LLMscited by 6
  4. 2025Mechanistic Anomaly Detection for "Quirky" Language Modelscited by 5
  5. 2025Be Careful When Fine-tuning On Open-Source LLMs: Your Fine-tuning Data Could Be Secretly Stolen!cited by 5
  6. 2025Propaganda AI: An Analysis of Semantic Divergence in Large Language Modelscited by 3
  7. 2025TeleLoRA: Teleporting Model-Specific Alignment Across LLMscited by 1
  8. 2025LLM in the Middle: A Systematic Review of Threats and Mitigations to Real-World LLM-based Systemscited by 1
  9. 2025Key-Conditioned Orthonormal Transform Gating (K-OTG): Multi-Key Access Control with Hidden-State Scrambling for LoRA-Tuned Modelscited by 1
  10. 2026Toward Reliable, Safe, and Secure LLMs for Scientific Applicationscited by 1
  11. 2026BadLLM-TG: A Backdoor Defender powered by LLM Trigger Generatorcited by 0
  12. 2026DebugLM: Learning Traceable Training Data Provenance for LLMscited by 0
3 more
  1. 2026Be Kind, Rewrite: Benign Projections via Rewriting Defend Against LLM Data Poisoning Attackscited by 0
  2. 2026Defending Against Harmful Supervision Hidden in Benign Samplescited by 0
  3. 2026An Empirical Study of Output-to-Input Loops for Black-Box Backdoor Detection in Fine-Tuned Open-Weight LLMscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024A Survey of Recent Backdoor Attacks and Defenses in Large Language Models
  2. 2025A Survey on Backdoor Threats in Large Language Models (LLMs): Attacks, Defenses, and Evaluations
  3. 2026Merging Triggers, Breaking Backdoors: Defensive Poisoning for Instruction-Tuned Language Models
  4. 2023A Comprehensive Overview of Backdoor Attacks in Large Language Models within Communication Networks
  5. 2025From Poisoned to Aware: Fostering Backdoor Self-Awareness in LLMs
  6. 2024Simulate and Eliminate: Revoke Backdoors for Generative Large Language Models
  7. 2024BackdoorLLM: A Comprehensive Benchmark for Backdoor Attacks and Defenses on Large Language Models
  8. 2025Data Security in Large Language Models: Risks, Defense, and Directions