mlsanityData to 14 Oct 2026

Hidden Killer: Invisible Textual Backdoor Attacks with Syntactic Trigger

arXiv 2105.12400 ↗First posted 26 May 2021cs.CL cs.CRcited by 222

On the map: LLM safety and security → Backdoor and data poisoning attacks

Abstract (arXiv)

Backdoor attacks are a kind of insidious security threat against machine learning models. After being injected with a backdoor in training, the victim model will produce adversary-specified outputs on the inputs embedded with predesigned triggers but behave properly on normal inputs during inference. As a sort of emergent attack, backdoor attacks in natural language processing (NLP) are investigated insufficiently. As far as we know, almost all existing textual backdoor attack methods insert additional contents into normal samples as triggers, which causes the trigger-embedded samples to be detected and the backdoor attacks to be blocked without much effort. In this paper, we propose to use the syntactic structure as the trigger in textual backdoor attacks. We conduct extensive experiments to demonstrate that the syntactic trigger-based attack method can achieve comparable attack performance (almost 100% success rate) to the insertion-based methods but possesses much higher invisibility and stronger resistance to defenses. These results also reveal the significant insidiousness and harmfulness of textual backdoor attacks. All the code and data of this paper can be obtained at https://github.com/thunlp/HiddenKiller.

Key facts. Cited by 222 papers in the map (data through 14 Oct 2026). Placed in Backdoor and data poisoning attacks, one of 12 directions in LLM safety and security. Builds on 28 of its 44 references that are in the map. First posted on arXiv 26 May 2021.

Builds on

28 of 44 references.

References · 28

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  5. 2013Intriguing properties of neural networkscited by 5,155
  6. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,548
  7. 2015Character-level Convolutional Networks for Text Classificationcited by 2,330
  8. 2017Get To The Point: Summarization with Pointer-Generator Networkscited by 1,292
  9. 2017BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chaincited by 1,106
  10. 2017Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoningcited by 1,055
  11. 2018Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networkscited by 580
  12. 2019Hidden Trigger Backdoor Attackscited by 371
16 more
  1. 2020Reflection Backdoor: A Natural Backdoor Attack on Deep Neural Networkscited by 349
  2. 2020Weight Poisoning Attacks on Pre-trained Modelscited by 341
  3. 2017Certified Defenses for Data Poisoning Attackscited by 303
  4. 2018Adversarial Example Generation with Syntactically Controlled Paraphrase Networkscited by 286
  5. 2019Predicting the Type and Target of Offensive Posts in Social Mediacited by 271
  6. 2020ONION: A Simple and Effective Defense Against Textual Backdoor Attackscited by 263
  7. 2019Adversarial Attacks and Defenses in Images, Graphs and Text: A Reviewcited by 237
  8. 2019A backdoor attack against LSTM-based text classification systemscited by 227
  9. 2020Clean-Label Backdoor Attacks on Video Recognition Modelscited by 169
  10. 2019Universal Litmus Patterns: Revealing Backdoor Attacks in CNNscited by 146
  11. 2018Backdoor Embedding in Convolutional Neural Network Models via Invisible Perturbationcited by 134
  12. 2019Robust Anomaly Detection and Backdoor Attack Detection Via Differential Privacycited by 95
  13. 2020Trojaning Language Models for Fun and Profitcited by 90
  14. 2020Mitigating backdoor attacks in LSTM-based Text Classification Systems by Backdoor Keyword Identificationcited by 86
  15. 2019Defending Neural Backdoors via Generative Distribution Modelingcited by 84
  16. 2017Generative Poisoning Attack Method Against Neural Networkscited by 52

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 10; 2022: 24; 2023: 41; 2024: 62; 2025: 46; 2026: 39.

222 papers.

Sort:

By area:

  1. 2024Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Trainingcited by 529
  2. 2023Poisoning Web-Scale Training Datasets is Practicalcited by 290
  3. 2023Large Language Model Alignment: A Surveycited by 243
  4. 2024Security and Privacy Challenges of Large Language Models: A Surveycited by 186
  5. 2021Mind the Style of Text! Adversarial and Backdoor Attacks Based on Text Style Transfercited by 158
  6. 2023Tensor Trust: Interpretable Prompt Injection Attacks from an Online Gamecited by 145
  7. 2024Watch Out for Your Agents! Investigating Backdoor Threats to LLM-Based Agentscited by 137
  8. 2022BackdoorBench: A Comprehensive Benchmark of Backdoor Learningcited by 128
  9. 2024BadAgent: Inserting and Activating Backdoor Attacks in LLM Agentscited by 115
  10. 2023Instructions as Backdoors: Backdoor Vulnerabilities of Instruction Tuning for Large Language Modelscited by 110
  11. 2021RAP: Robustness-Aware Perturbations for Defending against Backdoor Attacks on NLP Modelscited by 104
  12. 2023Prompt as Triggers for Backdoor Attack: Examining the Vulnerability in Language Modelscited by 94
38 more
  1. 2021Turn the Combination Lock: Learnable Textual Backdoor Attacks via Word Substitutioncited by 89
  2. 2021BadPre: Task-agnostic Backdoor Attacks to Pre-trained NLP Foundation Modelscited by 85
  3. 2022A Unified Evaluation of Textual Backdoor Learning: Frameworks and Benchmarkscited by 74
  4. 2025Poisoning Attacks on LLMs Require a Near-constant Number of Poison Samplescited by 72
  5. 2021Triggerless Backdoor Attack for NLP Tasks with Clean Labelscited by 66
  6. 2024Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systemscited by 66
  7. 2025Safety in Large Reasoning Models: A Surveycited by 64
  8. 2022BadPrompt: Backdoor Attacks on Continuous Promptscited by 63
  9. 2023TrojanPuzzle: Covertly Poisoning Code-Suggestion Modelscited by 60
  10. 2022Exploring the Universal Vulnerability of Prompt-based Learning Paradigmcited by 57
  11. 2022Fine-mixing: Mitigating Backdoors in Fine-tuned Language Modelscited by 57
  12. 2022BITE: Textual Backdoor Attacks with Iterative Trigger Injectioncited by 53
  13. 2024Universal Vulnerabilities in Large Language Models: Backdoor Attacks for In-context Learningcited by 53
  14. 2021CoProtector: Protect Open-Source Code against Unauthorized Training Usage with Data Poisoningcited by 47
  15. 2022Constrained Optimization with Dynamic Bound-scaling for Effective NLPBackdoor Defensecited by 47
  16. 2023UNICORN: A Unified Backdoor Trigger Inversion Frameworkcited by 46
  17. 2024Operationalizing a Threat Model for Red-Teaming Large Language Models (LLMs)cited by 46
  18. 2021An Overview of Backdoor Attacks Against Deep Neural Networks and Possible Defencescited by 42
  19. 2022Rethinking the Reverse-engineering of Trojan Triggerscited by 39
  20. 2023Backdoor Attacks and Countermeasures in Natural Language Processing Models: A Comprehensive Security Reviewcited by 39
  21. 2023NOTABLE: Transferable Backdoor Attacks Against Prompt-based NLP Modelscited by 37
  22. 2024Securing Large Language Models: Threats, Vulnerabilities and Responsible Practicescited by 37
  23. 2024A Survey of Recent Backdoor Attacks and Defenses in Large Language Modelscited by 36
  24. 2024Best-of-Venom: Attacking RLHF by Injecting Poisoned Preference Datacited by 35
  25. 2021Defending Against Backdoor Attacks in Natural Language Generationcited by 33
  26. 2023Defending Pre-trained Language Models as Few-shot Learners against Backdoor Attackscited by 33
  27. 2024PoisonBench: Assessing Large Language Model Vulnerability to Data Poisoningcited by 33
  28. 2024CROW: Eliminating Backdoors from Large Language Models via Internal Consistency Regularizationcited by 33
  29. 2023Attention-Enhancing Backdoor Attacks Against BERT-based Modelscited by 32
  30. 2023ChatGPT as an Attack Tool: Stealthy Textual Backdoor Attack via Blackbox Generative Model Triggercited by 31
  31. 2023TrojLLM: A Black-box Trojan Prompt Attack on Large Language Modelscited by 31
  32. 2023The Trickle-down Impact of Reward (In-)consistency on RLHFcited by 29
  33. 2023TrojText: Test-time Invisible Textual Trojan Insertioncited by 28
  34. 2023Watermarking Text Data on Large Language Models for Dataset Copyrightcited by 27
  35. 2023From Shortcuts to Triggers: Backdoor Defense with Denoised PoEcited by 27
  36. 2024Humanizing Machine-Generated Content: Evading AI-Text Detection through Adversarial Attackcited by 27
  37. 2024Unlearning Backdoor Attacks for LLMs with Weak-to-Strong Knowledge Distillationcited by 27
  38. 2026BadSkill: Backdoor Attacks on Agent Skills via Model-in-Skill Poisoningcited by 27

50 most cited of 222 All 222 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2021Hidden Backdoors in Human-Centric Language Models
  2. 2023Training-free Lexical Backdoor Attacks on Language Models
  3. 2025Trigger Where It Hurts: Unveiling Hidden Backdoors through Sensitivity with Sensitron ↗
  4. 2022Detecting Backdoors in Deep Text Classifiers
  5. 2021Backdoor Pre-trained Models Can Transfer to All
  6. 2024Exploiting the Vulnerability of Large Language Models via Defense-Aware Architectural Backdoor
  7. 2023Demystifying Poisoning Backdoor Attacks from a Statistical Perspective
  8. 2023Composite Backdoor Attacks Against Large Language Models