# Hidden Killer: Invisible Textual Backdoor Attacks with Syntactic Trigger

arXiv: https://arxiv.org/abs/2105.12400 · HTML: https://mlsanity.com/paper/2105.12400/

## Key facts

- Cited by 222 papers in the map (data through 14 Oct 2026).
- Placed in Backdoor and data poisoning attacks, one of 12 directions in LLM safety and security.
- Builds on 28 of its 44 references that are in the map.
- First posted on arXiv 26 May 2021.

## Placed in

- LLM safety and security / [Backdoor and data poisoning attacks](https://mlsanity.com/safety/backdoor-and-data-poisoning-attacks/)

## Builds on (most cited first, 28 of 28)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Character-level Convolutional Networks for Text Classification](https://mlsanity.com/paper/1509.01626/) (2015)
- [Get To The Point: Summarization with Pointer-Generator Networks](https://mlsanity.com/paper/1704.04368/) (2017)
- [BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain](https://mlsanity.com/paper/1708.06733/) (2017)
- [Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning](https://mlsanity.com/paper/1712.05526/) (2017)
- [Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networks](https://mlsanity.com/paper/1805.12185/) (2018)
- [Hidden Trigger Backdoor Attacks](https://mlsanity.com/paper/1910.00033/) (2019)
- [Reflection Backdoor: A Natural Backdoor Attack on Deep Neural Networks](https://mlsanity.com/paper/2007.02343/) (2020)
- [Weight Poisoning Attacks on Pre-trained Models](https://mlsanity.com/paper/2004.06660/) (2020)
- [Certified Defenses for Data Poisoning Attacks](https://mlsanity.com/paper/1706.03691/) (2017)
- [Adversarial Example Generation with Syntactically Controlled Paraphrase Networks](https://mlsanity.com/paper/1804.06059/) (2018)
- [Predicting the Type and Target of Offensive Posts in Social Media](https://mlsanity.com/paper/1902.09666/) (2019)
- [ONION: A Simple and Effective Defense Against Textual Backdoor Attacks](https://mlsanity.com/paper/2011.10369/) (2020)
- [Adversarial Attacks and Defenses in Images, Graphs and Text: A Review](https://mlsanity.com/paper/1909.08072/) (2019)
- [A backdoor attack against LSTM-based text classification systems](https://mlsanity.com/paper/1905.12457/) (2019)
- [Clean-Label Backdoor Attacks on Video Recognition Models](https://mlsanity.com/paper/2003.03030/) (2020)
- [Universal Litmus Patterns: Revealing Backdoor Attacks in CNNs](https://mlsanity.com/paper/1906.10842/) (2019)
- [Backdoor Embedding in Convolutional Neural Network Models via Invisible Perturbation](https://mlsanity.com/paper/1808.10307/) (2018)
- [Robust Anomaly Detection and Backdoor Attack Detection Via Differential Privacy](https://mlsanity.com/paper/1911.07116/) (2019)
- [Trojaning Language Models for Fun and Profit](https://mlsanity.com/paper/2008.00312/) (2020)
- [Mitigating backdoor attacks in LSTM-based Text Classification Systems by Backdoor Keyword Identification](https://mlsanity.com/paper/2007.12070/) (2020)
- [Defending Neural Backdoors via Generative Distribution Modeling](https://mlsanity.com/paper/1910.04749/) (2019)
- [Generative Poisoning Attack Method Against Neural Networks](https://mlsanity.com/paper/1703.01340/) (2017)

## Built on by (most cited first, 30 of 222)

- [Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training](https://mlsanity.com/paper/2401.05566/) (2024)
- [Poisoning Web-Scale Training Datasets is Practical](https://mlsanity.com/paper/2302.10149/) (2023)
- [Large Language Model Alignment: A Survey](https://mlsanity.com/paper/2309.15025/) (2023)
- [Security and Privacy Challenges of Large Language Models: A Survey](https://mlsanity.com/paper/2402.00888/) (2024)
- [Mind the Style of Text! Adversarial and Backdoor Attacks Based on Text Style Transfer](https://mlsanity.com/paper/2110.07139/) (2021)
- [Tensor Trust: Interpretable Prompt Injection Attacks from an Online Game](https://mlsanity.com/paper/2311.01011/) (2023)
- [Watch Out for Your Agents! Investigating Backdoor Threats to LLM-Based Agents](https://mlsanity.com/paper/2402.11208/) (2024)
- [BackdoorBench: A Comprehensive Benchmark of Backdoor Learning](https://mlsanity.com/paper/2206.12654/) (2022)
- [BadAgent: Inserting and Activating Backdoor Attacks in LLM Agents](https://mlsanity.com/paper/2406.03007/) (2024)
- [Instructions as Backdoors: Backdoor Vulnerabilities of Instruction Tuning for Large Language Models](https://mlsanity.com/paper/2305.14710/) (2023)
- [RAP: Robustness-Aware Perturbations for Defending against Backdoor Attacks on NLP Models](https://mlsanity.com/paper/2110.07831/) (2021)
- [Prompt as Triggers for Backdoor Attack: Examining the Vulnerability in Language Models](https://mlsanity.com/paper/2305.01219/) (2023)
- [Turn the Combination Lock: Learnable Textual Backdoor Attacks via Word Substitution](https://mlsanity.com/paper/2106.06361/) (2021)
- [BadPre: Task-agnostic Backdoor Attacks to Pre-trained NLP Foundation Models](https://mlsanity.com/paper/2110.02467/) (2021)
- [A Unified Evaluation of Textual Backdoor Learning: Frameworks and Benchmarks](https://mlsanity.com/paper/2206.08514/) (2022)
- [Poisoning Attacks on LLMs Require a Near-constant Number of Poison Samples](https://mlsanity.com/paper/2510.07192/) (2025)
- [Triggerless Backdoor Attack for NLP Tasks with Clean Labels](https://mlsanity.com/paper/2111.07970/) (2021)
- [Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems](https://mlsanity.com/paper/2401.05778/) (2024)
- [Safety in Large Reasoning Models: A Survey](https://mlsanity.com/paper/2504.17704/) (2025)
- [BadPrompt: Backdoor Attacks on Continuous Prompts](https://mlsanity.com/paper/2211.14719/) (2022)
- [TrojanPuzzle: Covertly Poisoning Code-Suggestion Models](https://mlsanity.com/paper/2301.02344/) (2023)
- [Exploring the Universal Vulnerability of Prompt-based Learning Paradigm](https://mlsanity.com/paper/2204.05239/) (2022)
- [Fine-mixing: Mitigating Backdoors in Fine-tuned Language Models](https://mlsanity.com/paper/2210.09545/) (2022)
- [BITE: Textual Backdoor Attacks with Iterative Trigger Injection](https://mlsanity.com/paper/2205.12700/) (2022)
- [Universal Vulnerabilities in Large Language Models: Backdoor Attacks for In-context Learning](https://mlsanity.com/paper/2401.05949/) (2024)
- [CoProtector: Protect Open-Source Code against Unauthorized Training Usage with Data Poisoning](https://mlsanity.com/paper/2110.12925/) (2021)
- [Constrained Optimization with Dynamic Bound-scaling for Effective NLPBackdoor Defense](https://mlsanity.com/paper/2202.05749/) (2022)
- [UNICORN: A Unified Backdoor Trigger Inversion Framework](https://mlsanity.com/paper/2304.02786/) (2023)
- [Operationalizing a Threat Model for Red-Teaming Large Language Models (LLMs)](https://mlsanity.com/paper/2407.14937/) (2024)
- [An Overview of Backdoor Attacks Against Deep Neural Networks and Possible Defences](https://mlsanity.com/paper/2111.08429/) (2021)

## Abstract (arXiv)

Backdoor attacks are a kind of insidious security threat against machine learning models. After being injected with a backdoor in training, the victim model will produce adversary-specified outputs on the inputs embedded with predesigned triggers but behave properly on normal inputs during inference. As a sort of emergent attack, backdoor attacks in natural language processing (NLP) are investigated insufficiently. As far as we know, almost all existing textual backdoor attack methods insert additional contents into normal samples as triggers, which causes the trigger-embedded samples to be detected and the backdoor attacks to be blocked without much effort. In this paper, we propose to use the syntactic structure as the trigger in textual backdoor attacks. We conduct extensive experiments to demonstrate that the syntactic trigger-based attack method can achieve comparable attack performance (almost 100% success rate) to the insertion-based methods but possesses much higher invisibility and stronger resistance to defenses. These results also reveal the significant insidiousness and harmfulness of textual backdoor attacks. All the code and data of this paper can be obtained at https://github.com/thunlp/HiddenKiller.