mlsanityData to 7 Oct 2026

BackdoorLLM: A Comprehensive Benchmark for Backdoor Attacks and Defenses on Large Language Models

arXiv 2408.12798 ↗First posted 23 Aug 2024cs.AIcited by 44

On the map: LLM safety and security → Backdoor and data poisoning attacks

Abstract (arXiv)

Generative large language models (LLMs) have achieved state-of-the-art results on a wide range of tasks, yet they remain susceptible to backdoor attacks: carefully crafted triggers in the input can manipulate the model to produce adversary-specified outputs. While prior research has predominantly focused on backdoor risks in vision and classification settings, the vulnerability of LLMs in open-ended text generation remains underexplored. To fill this gap, we introduce BackdoorLLM (Our BackdoorLLM benchmark was awarded First Prize in the SafetyBench competition, https://www.mlsafety.org/safebench/winners, organized by the Center for AI Safety, https://safe.ai/.), the first comprehensive benchmark for systematically evaluating backdoor threats in text-generation LLMs. BackdoorLLM provides: (i) a unified repository of benchmarks with a standardized training and evaluation pipeline; (ii) a diverse suite of attack modalities, including data poisoning, weight poisoning, hidden-state manipulation, and chain-of-thought hijacking; (iii) over 200 experiments spanning 8 distinct attack strategies, 7 real-world scenarios, and 6 model architectures; (iv) key insights into the factors that govern backdoor effectiveness and failure modes in LLMs; and (v) a defense toolkit encompassing 7 representative mitigation techniques. Our code and datasets are available at https://github.com/bboylyg/BackdoorLLM. We will continuously incorporate emerging attack and defense methodologies to support the research in advancing the safety and reliability of LLMs.

Key facts. Cited by 44 papers in the map (data through 7 Oct 2026). Placed in Backdoor and data poisoning attacks, one of 12 directions in LLM safety and security. Builds on 39 of its 51 references that are in the map. First posted on arXiv 23 Aug 2024.

Builds on

39 of 51 references.

Closest ancestors · 9

Same direction, newest first

  1. 2024BEEAR: Embedding-based Adversarial Removal of Safety Backdoors in Instruction-tuned Language Models24 Jun
  2. 2024CleanGen: Mitigating Backdoor Attacks for Generation Tasks in Large Language Models18 Jun
  3. 2024BadEdit: Backdooring large language models by model editing20 Mar
  4. 2024Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training10 Jan
  5. 2023BadCLIP: Trigger-Aware Prompt Learning for Backdoor Attacks on CLIP26 Nov
  6. 2023Composite Backdoor Attacks Against Large Language Models11 Oct
  7. 2023Backdooring Instruction-Tuned Large Language Models with Virtual Prompt Injection31 Jul
  8. 2023Instructions as Backdoors: Backdoor Vulnerabilities of Instruction Tuning for Large Language Models24 May
  9. 2022BadPrompt: Backdoor Attacks on Continuous Prompts27 Nov

Other references · 30

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  5. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  6. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  7. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  8. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  9. 2023Mistral 7Bcited by 3,065
  10. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  11. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  12. 2015Character-level Convolutional Networks for Text Classificationcited by 2,293
18 more
  1. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,793
  2. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,106
  3. 2017BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chaincited by 1,052
  4. 2023A Simple and Effective Pruning Approach for Large Language Modelscited by 776
  5. 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 736
  6. 2021BOLD: Dataset and Metrics for Measuring Biases in Open-Ended Language Generationcited by 410
  7. 2021A Diverse Corpus for Evaluating and Developing English Math Word Problem Solverscited by 314
  8. 2021Neural Attention Distillation: Erasing Backdoor Triggers from Deep Neural Networkscited by 313
  9. 2021Anti-Backdoor Learning: Training Clean Models on Poisoned Datacited by 258
  10. 2020BadNL: Backdoor Attacks against NLP Models with Semantic-preserving Improvementscited by 191
  11. 2023Multilingual Machine Translation with Large Language Models: Empirical Results and Analysiscited by 156
  12. 2022BackdoorBench: A Comprehensive Benchmark of Backdoor Learningcited by 124
  13. 2023Universal Jailbreak Backdoors from Poisoned Human Feedbackcited by 120
  14. 2024A Thorough Examination of Decoding Methods in the Era of LLMscited by 112
  15. 2023Reconstructive Neuron Pruning for Backdoor Defensecited by 56
  16. 2023InfoPrompt: Information-Theoretic Soft Prompt Tuning for Natural Language Understandingcited by 29
  17. 2024Competition Report: Finding Universal Jailbreak Backdoors in Aligned LLMscited by 27
  18. 2023An Empirical Study of Metrics to Measure Representational Harms in Pre-Trained Language Modelscited by 24

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 13; 2026: 28.

44 papers.

Sort:

By area:

  1. 2024Navigating the Risks: A Survey of Security, Privacy, and Ethics Threats in LLM-Based Agentscited by 63
  2. 2025Safety at Scale: A Comprehensive Survey of Large Model and Agent Safetycited by 63
  3. 2025A Comprehensive Survey on Trustworthiness in Reasoning with Large Language Modelscited by 22
  4. 2026BackdoorAgent: A Unified Framework for Backdoor Attacks on LLM-based Agentscited by 12
  5. 2026A Systematic Survey of Security Threats and Defenses in LLM-Based AI Agents: A Layered Attack Surface Frameworkcited by 9
  6. 2025Watch the Weights: Unsupervised monitoring and control of fine-tuned LLMscited by 7
  7. 2025AutoBackdoor: Automating Backdoor Attacks via LLM Agentscited by 7
  8. 2025When Style Breaks Safety: Defending LLMs Against Superficial Style Alignmentcited by 6
  9. 2025BadReasoner: Planting Tunable Overthinking Backdoors into Large Reasoning Models for Fun or Profitcited by 6
  10. 2026Purifying Generative LLMs from Backdoors without Prior Knowledge or Clean Referencecited by 6
  11. 2024Model-Editing-Based Jailbreak against Safety-aligned Large Language Modelscited by 4
  12. 2025Where Did It Go Wrong? Attributing Undesirable LLM Behaviors via Representation Gradient Tracingcited by 4
32 more
  1. 2025Forgetting to Forget: Attention Sink as A Gateway for Backdooring LLM Unlearningcited by 4
  2. 2025Thought Purity: A Defense Framework For Chain-of-Thought Attackcited by 3
  3. 2025Pay Attention to the Triggers: Constructing Backdoors That Survive Distillationcited by 3
  4. 2026Trusting What You Cannot See: Auditable Fine-Tuning and Inference for Proprietary AIcited by 3
  5. 2024On the Validity of Traditional Vulnerability Scoring Systems for Adversarial Attacks against LLMscited by 2
  6. 2025MobiLLM: An Agentic AI Framework for Closed-Loop Threat Mitigation in 6G Open RANscited by 2
  7. 2026Self-Purification Mitigates Backdoors in Multimodal Diffusion Language Modelscited by 2
  8. 2025Key-Conditioned Orthonormal Transform Gating (K-OTG): Multi-Key Access Control with Hidden-State Scrambling for LoRA-Tuned Modelscited by 1
  9. 2026Backdooring Bias in Large Language Modelscited by 1
  10. 2026Toward Reliable, Safe, and Secure LLMs for Scientific Applicationscited by 1
  11. 2026Stealthy Backdoor Attacks against LLMs Based on Natural Style Triggerscited by 1
  12. 2026Layerwise Convergence Fingerprints for Runtime Misbehavior Detection in Large Language Modelscited by 1
  13. 2026PoisonForge: Task-Level Targeted Poisoning Benchmark for Instruction-Tuned LLMscited by 1
  14. 2026Security in the Fine-Tuning Lifecycle of Large Language Models: Threats, Defenses,Evaluation, and Future Directionscited by 1
  15. 2026RogueMerge: Robust and Unified Attacks against LLM Model Mergingcited by 1
  16. 2025Data Security in Large Language Models: Risks, Defense, and Directionscited by 0
  17. 2026Backdoor4Good: Benchmarking Beneficial Uses of Backdoors in LLMscited by 0
  18. 2026Trojan horse hunt in deep forecasting models: Insights from the European Space Agency competitioncited by 0
  19. 2026Secret Stealing Attacks on Local LLM Fine-Tuning through Supply-Chain Model Code Backdoorscited by 0
  20. 2026On the Privacy of LLMs: An Ablation Studycited by 0
  21. 2026Be Kind, Rewrite: Benign Projections via Rewriting Defend Against LLM Data Poisoning Attackscited by 0
  22. 2026Backdooring Masked Diffusion Language Modelscited by 0
  23. 2026Cordyceps: Covert Control Attacks on LLMs via Data Poisoningcited by 0
  24. 2026Curvature-Guided Module Localization for Low-Rank Detoxification of Backdoored Large Language Modelscited by 0
  25. 2026Cyber-Capable AI Agents: Vulnerabilities, Evaluation Containment, and Defensive Responsecited by 0
  26. 2026Compositional Threat Analysis of Latent Compromise in LLM Agent Systems: The Order 66 Scenariocited by 0
  27. 2026Backdoor Decontamination Dynamics in LLM Agentscited by 0
  28. 2026Not All Tokens Are Equal: Region-Aware Consistency Repair of Backdoors in MLLMscited by 0
  29. 2026SilentCall: Hidden Tool-Call Backdoors in Open-Weight Agents, and How to Catch Themcited by 0
  30. 2026Removing the NEEDLE in the Haystack: Backdoor Removal in LLMs via Weight Orthogonalisationcited by 0
  31. 2026Backdoor Containment via Expert Quarantine and Shutdown in LLMscited by 0
  32. 2026Backdoor Purification for LoRA-Tuned LLMs via Null-Space Projectioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Simulate and Eliminate: Revoke Backdoors for Generative Large Language Models
  2. 2024Mitigating Backdoor Threats to Large Language Models: Advancement and Challenges
  3. 2024TrojanRAG: Retrieval-Augmented Generation Can Be Backdoor Driver in Large Language Models
  4. 2026BadDLM: Backdooring Diffusion Language Models with Diverse Targets
  5. 2025BackdoorVLM: A Benchmark for Backdoor Attacks and Defenses on Vision-Language Models
  6. 2024BadChain: Backdoor Chain-of-Thought Prompting for Large Language Models
  7. 2024Rethinking Backdoor Detection Evaluation for Language Models
  8. 2023Survey of Vulnerabilities in Large Language Models Revealed by Adversarial Attacks