mlsanityData to 7 Oct 2026

Rethinking Backdoor Detection Evaluation for Language Models

arXiv 2409.00399 ↗First posted 31 Aug 2024cs.CL cs.CRcited by 6

On the map: LLM reasoning → Reasoning for agents, planning and code

also in 1 more
Abstract (arXiv)

Backdoor attacks, in which a model behaves maliciously when given an attacker-specified trigger, pose a major security risk for practitioners who depend on publicly released language models. As a countermeasure, backdoor detection methods aim to detect whether a released model contains a backdoor. While existing backdoor detection methods have high accuracy in detecting backdoored models on standard benchmarks, it is unclear whether they can robustly identify backdoors in the wild. In this paper, we examine the robustness of backdoor detectors by manipulating different factors during backdoor planting. We find that the success of existing methods based on trigger inversion or meta classifiers highly depends on how intensely the model is trained on poisoned data. Specifically, backdoors planted with more aggressive or more conservative training are significantly more difficult to detect than the default ones. Our results highlight a lack of robustness of existing backdoor detectors and the limitations in current benchmark construction.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 44 of its 60 references that are in the map. First posted on arXiv 31 Aug 2024.

Builds on

44 of 60 references.

References · 44

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  4. 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,259
  5. 2017Visualizing the Loss Landscape of Neural Netscited by 1,126
  6. 2017BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chaincited by 1,052
  7. 2017Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoningcited by 1,012
  8. 2018Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networkscited by 550
  9. 2021Datasets: A Community Library for Natural Language Processingcited by 526
  10. 2024Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Trainingcited by 478
  11. 2020Backdoor Learning: A Surveycited by 412
  12. 2020Weight Poisoning Attacks on Pre-trained Modelscited by 318
32 more
  1. 2020ONION: A Simple and Effective Defense Against Textual Backdoor Attackscited by 231
  2. 2019A backdoor attack against LSTM-based text classification systemscited by 214
  3. 2021Hidden Killer: Invisible Textual Backdoor Attacks with Syntactic Triggercited by 203
  4. 2018Hate Speech Dataset from a White Supremacy Forumcited by 196
  5. 2019Detecting AI Trojans Using Meta Neural Analysiscited by 192
  6. 2020Dataset Security for Machine Learning: Data Poisoning, Backdoor Attacks, and Defensescited by 171
  7. 2021Mind the Style of Text! Adversarial and Backdoor Attacks Based on Text Style Transfercited by 147
  8. 2019Universal Litmus Patterns: Revealing Backdoor Attacks in CNNscited by 143
  9. 2023Backdooring Instruction-Tuned Large Language Models with Virtual Prompt Injectioncited by 143
  10. 2022BackdoorBench: A Comprehensive Benchmark of Backdoor Learningcited by 124
  11. 2023Universal Jailbreak Backdoors from Poisoned Human Feedbackcited by 120
  12. 2021RAP: Robustness-Aware Perturbations for Defending against Backdoor Attacks on NLP Modelscited by 97
  13. 2024BadEdit: Backdooring large language models by model editingcited by 84
  14. 2020Mitigating backdoor attacks in LSTM-based Text Classification Systems by Backdoor Keyword Identificationcited by 80
  15. 2021Backdoor Pre-trained Models Can Transfer to Allcited by 79
  16. 2021BadPre: Task-agnostic Backdoor Attacks to Pre-trained NLP Foundation Modelscited by 76
  17. 2022A Unified Evaluation of Textual Backdoor Learning: Frameworks and Benchmarkscited by 70
  18. 2021T-Miner: A Generative Approach to Defend Against Trojan Attacks on DNN-based Text Classificationcited by 56
  19. 2022Planting Undetectable Backdoors in Machine Learning Modelscited by 55
  20. 2022A Study of the Attention Abnormality in Trojaned BERTscited by 47
  21. 2022BITE: Textual Backdoor Attacks with Iterative Trigger Injectioncited by 47
  22. 2022Constrained Optimization with Dynamic Bound-scaling for Effective NLPBackdoor Defensecited by 44
  23. 2020The TrojAI Software Framework: An OpenSource tool for Embedding Trojans into Deep Learning Modelscited by 41
  24. 2023Training-free Lexical Backdoor Attacks on Language Modelscited by 27
  25. 2023From Shortcuts to Triggers: Backdoor Defense with Denoised PoEcited by 26
  26. 2024Here's a Free Lunch: Sanitizing Backdoored Models with Model Mergecited by 21
  27. 2023Test-time Backdoor Mitigation for Black-Box Large Language Models with Defensive Demonstrationscited by 20
  28. 2023Mitigating Backdoor Poisoning Attacks through the Lens of Spurious Correlationcited by 19
  29. 2022Expose Backdoors on the Way: A Feature-Based Efficient Defense against Textual Backdoor Attackscited by 18
  30. 2021Trojan Signatures in DNN Weightscited by 15
  31. 2023Gradient Shaping: Enhancing Backdoor Attack Against Reverse Engineeringcited by 4
  32. 2024On the (In)feasibility of ML Backdoor Detection as an Hypothesis Testing Problemcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 1; 2026: 4.

6 papers.

  1. 2024Unlearning Backdoor Attacks for LLMs with Weak-to-Strong Knowledge Distillationcited by 21
  2. 2026Triaging Threats to Specialized Guardrailscited by 1
  3. 2025RN-F: A Novel Approach for Mitigating Contaminated Data in Large Language Modelscited by 0
  4. 2026When Emotion Becomes Trigger: Emotion-style dynamic Backdoor Attack Parasitising Large Language Modelscited by 0
  5. 2026Be Kind, Rewrite: Benign Projections via Rewriting Defend Against LLM Data Poisoning Attackscited by 0
  6. 2026Decision-Level Hijacking: Injecting Cognitive Bias into Large Language Models via Bit-Flip Attackscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024BackdoorLLM: A Comprehensive Benchmark for Backdoor Attacks and Defenses on Large Language Models
  2. 2026Patcher: Post-Hoc Patching of Backdoored Large Language Models
  3. 2024TrojanRAG: Retrieval-Augmented Generation Can Be Backdoor Driver in Large Language Models
  4. 2025Uncovering and Aligning Anomalous Attention Heads to Defend Against NLP Backdoor Attacks
  5. 2024Chain-of-Scrutiny: Detecting Backdoor Attacks for Large Language Models
  6. 2024Mitigating Backdoor Threats to Large Language Models: Advancement and Challenges
  7. 2026BadDLM: Backdooring Diffusion Language Models with Diverse Targets
  8. 2025Probe before You Talk: Towards Black-box Defense against Backdoor Unalignment for Large Language Models