mlsanityData to 14 Oct 2026

Hidden Backdoors in Human-Centric Language Models

arXiv 2105.00164 ↗First posted 1 May 2021cs.CL cs.CRcited by 90

On the map: LLM safety and security → Backdoor and data poisoning attacks

Abstract (arXiv)

Natural language processing (NLP) systems have been proven to be vulnerable to backdoor attacks, whereby hidden features (backdoors) are trained into a language model and may only be activated by specific inputs (called triggers), to trick the model into producing unexpected behaviors. In this paper, we create covert and natural triggers for textual backdoor attacks, textithidden backdoors, where triggers can fool both modern language models and human inspection. We deploy our hidden backdoors through two state-of-the-art trigger embedding methods. The first approach via homograph replacement, embeds the trigger into deep neural networks through the visual spoofing of lookalike character replacement. The second approach uses subtle differences between text generated by language models and real natural text to produce trigger sentences with correct grammar and high fluency. We demonstrate that the proposed hidden backdoors can be effective across three downstream security-critical NLP tasks, representative of modern human-centric NLP systems, including toxic comment detection, neural machine translation (NMT), and question answering (QA). Our two hidden backdoor attacks can achieve an Attack Success Rate (ASR) of at least 97% with an injection rate of only 3% in toxic comment detection, 95.1% ASR in NMT with less than 0.5% injected data, and finally 91.12% ASR against QA updated with only 27 poisoning data samples on a model previously trained with 92,024 samples (0.029%). We are able to demonstrate the adversary's high success rate of attacks, while maintaining functionality for regular users, with triggers inconspicuous by the human administrators.

Key facts. Cited by 90 papers in the map (data through 14 Oct 2026). Placed in Backdoor and data poisoning attacks, one of 12 directions in LLM safety and security. Builds on 46 of its 81 references that are in the map. First posted on arXiv 1 May 2021.

Builds on

46 of 81 references.

References · 46

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
  4. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,918
  5. 2020Extracting Training Data from Large Language Modelscited by 2,189
  6. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,799
  7. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,554
  8. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,488
  9. 2019Plug and Play Language Models: A Simple Approach to Controlled Text Generationcited by 799
  10. 2016Universal adversarial perturbationscited by 784
  11. 2019Universal Adversarial Triggers for Attacking and Analyzing NLPcited by 722
  12. 2019STRIP: A Defence Against Trojan Attacks on Deep Neural Networkscited by 459
34 more
  1. 2018TextBugger: Generating Adversarial Text Against Real-world Applicationscited by 381
  2. 2020Weight Poisoning Attacks on Pre-trained Modelscited by 341
  3. 2019A backdoor attack against LSTM-based text classification systemscited by 227
  4. 2018Manipulating Machine Learning: Poisoning Attacks and Countermeasures for Regression Learningcited by 225
  5. 2019Detecting AI Trojans Using Meta Neural Analysiscited by 199
  6. 2020Blind Backdoors in Deep Learning Modelscited by 182
  7. 2019CCMatrix: Mining Billions of High-Quality Parallel Sentences on the WEBcited by 177
  8. 2020Dynamic Backdoor Attacks Against Machine Learning Modelscited by 151
  9. 2020Backdoor Attacks to Graph Neural Networkscited by 128
  10. 2019Demon in the Variant: Statistical Analysis of DNNs for Robust Backdoor Contamination Detectioncited by 124
  11. 2019TABOR: A Highly Accurate Approach to Inspecting and Restoring Trojan Backdoors in AI Systemscited by 122
  12. 2019TBT: Targeted Neural Network Attack with Bit Trojancited by 113
  13. 2020Graph Backdoorcited by 112
  14. 2020Deep Feature Space Trojan Attack of Neural Networks by Controlled Detoxificationcited by 95
  15. 2020Intrinsic Certified Robustness of Bagging against Data Poisoning Attackscited by 94
  16. 2020InfoBERT: Improving Robustness of Language Models from An Information Theoretic Perspectivecited by 92
  17. 2019Analyzing Information Leakage of Updates to Natural Language Modelscited by 90
  18. 2020Trojaning Language Models for Fun and Profitcited by 90
  19. 2017A General Framework for Adversarial Examples with Objectivescited by 84
  20. 2019Defending Neural Backdoors via Generative Distribution Modelingcited by 84
  21. 2020Imitation Attacks and Defenses for Black-box Machine Translation Systemscited by 81
  22. 2019Bypassing Backdoor Detection Algorithms in Deep Learningcited by 74
  23. 2021Data Poisoning Attacks to Deep Learning Based Recommender Systemscited by 43
  24. 2020Adversarial Semantic Collisionscited by 42
  25. 2019Gotta Catch 'Em All: Using Honeypots to Catch Adversarial Attacks on Neural Networkscited by 40
  26. 2019Data Poisoning Attacks to Local Differential Privacy Protocolscited by 26
  27. 2019The Audio Auditor: User-Level Membership Inference in Internet of Things Voice Servicescited by 25
  28. 2020Deep Learning Backdoorscited by 19
  29. 2020Don't Trigger Me! A Triggerless Backdoor Attack Against Deep Neural Networkscited by 19
  30. 2020Rapidly Bootstrapping a Question Answering Dataset for COVID-19cited by 14
  31. 2020With Great Dispersion Comes Greater Resilience: Efficient Poisoning Attacks and Defenses for Linear Regression Modelscited by 10
  32. 2018Detecting Homoglyph Attacks with a Siamese Neural Networkcited by 5
  33. 2018Detecting Outliers in Data with Correlated Measurescited by 4
  34. 2020A Targeted Attack on Black-Box Neural Machine Translation with Parallel Data Poisoningcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 3; 2022: 12; 2023: 28; 2024: 26; 2025: 13; 2026: 8.

90 papers.

Sort:

By area:

  1. 2023Large Language Model Alignment: A Surveycited by 243
  2. 2024BadAgent: Inserting and Activating Backdoor Attacks in LLM Agentscited by 115
  3. 2023Backdoor Attacks for In-Context Learning with Language Modelscited by 96
  4. 2023A Survey of Safety and Trustworthiness of Large Language Models through the Lens of Verification and Validationcited by 93
  5. 2023Text-to-Image Diffusion Models can be Easily Backdoored through Multimodal Data Poisoningcited by 92
  6. 2023When Large Language Models Meet Personalization: Perspectives of Challenges and Opportunitiescited by 90
  7. 2021Spinning Language Models: Risks of Propaganda-As-A-Service and Countermeasurescited by 73
  8. 2023Are You Copying My Model? Protecting the Copyright of Large Language Models for EaaS via Backdoor Watermarkcited by 67
  9. 2022The "Beatrix'' Resurrections: Robust Backdoor Detection via Gram Matricescited by 63
  10. 2022BadPrompt: Backdoor Attacks on Continuous Promptscited by 63
  11. 2023A Survey on Federated Unlearning: Challenges, Methods, and Future Directionscited by 63
  12. 2022Exploring the Universal Vulnerability of Prompt-based Learning Paradigmcited by 57
38 more
  1. 2022MM-BD: Post-Training Detection of Backdoor Attacks with Arbitrary Backdoor Pattern Types Using a Maximum Margin Statisticcited by 55
  2. 2023From Prompt Injections to SQL Injection Attacks: How Protected is Your LLM-Integrated Web Application?cited by 55
  3. 2023Stealthy Backdoor Attack for Code Modelscited by 49
  4. 2023Tricking LLMs into Disobedience: Formalizing, Analyzing, and Detecting Jailbreakscited by 49
  5. 2022Constrained Optimization with Dynamic Bound-scaling for Effective NLPBackdoor Defensecited by 47
  6. 2024Operationalizing a Threat Model for Red-Teaming Large Language Models (LLMs)cited by 46
  7. 2023A Comprehensive Overview of Backdoor Attacks in Large Language Models within Communication Networkscited by 42
  8. 2024Test-Time Backdoor Attacks on Multimodal Large Language Modelscited by 40
  9. 2023Backdoor Attacks and Countermeasures in Natural Language Processing Models: A Comprehensive Security Reviewcited by 39
  10. 2024Threats, Attacks, and Defenses in Machine Unlearning: A Surveycited by 39
  11. 2024A Survey of Recent Backdoor Attacks and Defenses in Large Language Modelscited by 36
  12. 2023Vulnerabilities in AI Code Generators: Exploring Targeted Data Poisoning Attackscited by 33
  13. 2023Training-free Lexical Backdoor Attacks on Language Modelscited by 31
  14. 2023ChatGPT as an Attack Tool: Stealthy Textual Backdoor Attack via Blackbox Generative Model Triggercited by 31
  15. 2025Agentic AI Security: Threats, Defenses, Evaluation, and Open Challengescited by 28
  16. 2021Data Hiding with Deep Learning: A Survey Unifying Digital Watermarking and Steganographycited by 24
  17. 2022Kallima: A Clean-label Framework for Textual Backdoor Attackscited by 24
  18. 2024Backdoored Retrievers for Prompt Injection Attacks on Retrieval Augmented Generation of Large Language Modelscited by 22
  19. 2024Double-I Watermark: Protecting Model Copyright for LLM Fine-tuningcited by 21
  20. 2024BadActs: A Universal Backdoor Defense in the Activation Spacecited by 21
  21. 2023Attacks in Adversarial Machine Learning: A Systematic Survey from the Life-cycle Perspectivecited by 20
  22. 2023ParaFuzz: An Interpretability-Driven Technique for Detecting Poisoned Samples in NLPcited by 19
  23. 2024Unbridled Icarus: A Survey of the Potential Perils of Image Inputs in Multimodal Large Language Model Securitycited by 19
  24. 2023UMD: Unsupervised Model Detection for X2X Backdoor Attackscited by 18
  25. 2023BDMMT: Backdoor Sample Detection for Language Models through Model Mutation Testingcited by 16
  26. 2024Defending Against Weight-Poisoning Backdoor Attacks for Parameter-Efficient Fine-Tuningcited by 16
  27. 2024Future Events as Backdoor Triggers: Investigating Temporal Vulnerabilities in LLMscited by 15
  28. 2025A Survey of Attacks on Large Language Modelscited by 15
  29. 2022A Survey on Backdoor Attack and Defense in Natural Language Processingcited by 14
  30. 2022Backdoor Vulnerabilities in Normally Trained Deep Learning Modelscited by 13
  31. 2024CLIBE: Detecting Dynamic Backdoors in Transformer-based NLP Modelscited by 13
  32. 2022Poison Attack and Defense on Deep Source Code Processing Modelscited by 11
  33. 2021Accumulative Poisoning Attacks on Real-time Datacited by 10
  34. 2024The Dark Side of Human Feedback: Poisoning Large Language Models via User Inputscited by 10
  35. 2023LMSanitator: Defending Prompt-Tuning Against Task-Agnostic Backdoorscited by 9
  36. 2023NCL: Textual Backdoor Defense Using Noise-augmented Contrastive Learningcited by 8
  37. 2024CL-Attack: Textual Backdoor Attacks via Cross-Lingual Triggerscited by 8
  38. 2025LLM Security: Vulnerabilities, Attacks, Defenses, and Countermeasurescited by 8

50 most cited of 90 All 90 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2021Hidden Killer: Invisible Textual Backdoor Attacks with Syntactic Trigger
  2. 2021Turn the Combination Lock: Learnable Textual Backdoor Attacks via Word Substitution
  3. 2024Large Language Models are Good Attackers: Efficient and Stealthy Textual Backdoor Attacks
  4. 2022Rethink the Evaluation for Attack Strength of Backdoor Attacks in Natural Language Processing
  5. 2025Unmasking Backdoors: An Explainable Defense via Gradient-Attention Anomaly Scoring for Pre-trained Language Models
  6. 2021BadPre: Task-agnostic Backdoor Attacks to Pre-trained NLP Foundation Models
  7. 2024Imperio: Language-Guided Backdoor Attacks for Arbitrary Model Control
  8. 2023Composite Backdoor Attacks Against Large Language Models