# Turn the Combination Lock: Learnable Textual Backdoor Attacks via Word Substitution

arXiv: https://arxiv.org/abs/2106.06361 · HTML: https://mlsanity.com/paper/2106.06361/

## Key facts

- Cited by 89 papers in the map (data through 14 Oct 2026).
- Placed in Backdoor and data poisoning attacks, one of 12 directions in LLM safety and security.
- Builds on 31 of its 49 references that are in the map.
- First posted on arXiv 11 Jun 2021.

## Placed in

- LLM safety and security / [Backdoor and data poisoning attacks](https://mlsanity.com/safety/backdoor-and-data-poisoning-attacks/)

## Builds on (most cited first, 30 of 31)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [Character-level Convolutional Networks for Text Classification](https://mlsanity.com/paper/1509.01626/) (2015)
- [BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain](https://mlsanity.com/paper/1708.06733/) (2017)
- [Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning](https://mlsanity.com/paper/1712.05526/) (2017)
- [Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailment](https://mlsanity.com/paper/1907.11932/) (2019)
- [Spectral Signatures in Backdoor Attacks](https://mlsanity.com/paper/1811.00636/) (2018)
- [Generating Natural Language Adversarial Examples](https://mlsanity.com/paper/1804.07998/) (2018)
- [Hidden Trigger Backdoor Attacks](https://mlsanity.com/paper/1910.00033/) (2019)
- [Turning Your Weakness Into a Strength: Watermarking Deep Neural Networks by Backdooring](https://mlsanity.com/paper/1802.04633/) (2018)
- [Reflection Backdoor: A Natural Backdoor Attack on Deep Neural Networks](https://mlsanity.com/paper/2007.02343/) (2020)
- [Weight Poisoning Attacks on Pre-trained Models](https://mlsanity.com/paper/2004.06660/) (2020)
- [Predicting the Type and Target of Offensive Posts in Social Media](https://mlsanity.com/paper/1902.09666/) (2019)
- [ONION: A Simple and Effective Defense Against Textual Backdoor Attacks](https://mlsanity.com/paper/2011.10369/) (2020)
- [A backdoor attack against LSTM-based text classification systems](https://mlsanity.com/paper/1905.12457/) (2019)
- [Hidden Killer: Invisible Textual Backdoor Attacks with Syntactic Trigger](https://mlsanity.com/paper/2105.12400/) (2021)
- [Detecting AI Trojans Using Meta Neural Analysis](https://mlsanity.com/paper/1910.03137/) (2019)
- [Clean-Label Backdoor Attacks on Video Recognition Models](https://mlsanity.com/paper/2003.03030/) (2020)
- [Neural Trojans](https://mlsanity.com/paper/1710.00942/) (2017)
- [Universal Litmus Patterns: Revealing Backdoor Attacks in CNNs](https://mlsanity.com/paper/1906.10842/) (2019)
- [Backdoor Embedding in Convolutional Neural Network Models via Invisible Perturbation](https://mlsanity.com/paper/1808.10307/) (2018)
- [How Does NLP Benefit Legal System: A Summary of Legal Artificial Intelligence](https://mlsanity.com/paper/2004.12158/) (2020)
- [Robust Anomaly Detection and Backdoor Attack Detection Via Differential Privacy](https://mlsanity.com/paper/1911.07116/) (2019)
- [Mitigating backdoor attacks in LSTM-based Text Classification Systems by Backdoor Keyword Identification](https://mlsanity.com/paper/2007.12070/) (2020)
- [On Certifying Robustness against Backdoor Attacks via Randomized Smoothing](https://mlsanity.com/paper/2002.11750/) (2020)
- [Towards Probabilistic Verification of Machine Unlearning](https://mlsanity.com/paper/2003.04247/) (2020)
- [Generating Fluent Adversarial Examples for Natural Languages](https://mlsanity.com/paper/2007.06174/) (2020)
- [OpenHowNet: An Open Sememe-based Lexical Knowledge Base](https://mlsanity.com/paper/1901.09957/) (2019)
- [Neural Lattice Language Models](https://mlsanity.com/paper/1803.05071/) (2018)
- [Modeling Semantic Compositionality with Sememe Knowledge](https://mlsanity.com/paper/1907.04744/) (2019)

## Built on by (most cited first, 30 of 89)

- [Mind the Style of Text! Adversarial and Backdoor Attacks Based on Text Style Transfer](https://mlsanity.com/paper/2110.07139/) (2021)
- [BackdoorBench: A Comprehensive Benchmark of Backdoor Learning](https://mlsanity.com/paper/2206.12654/) (2022)
- [RAP: Robustness-Aware Perturbations for Defending against Backdoor Attacks on NLP Models](https://mlsanity.com/paper/2110.07831/) (2021)
- [Prompt as Triggers for Backdoor Attack: Examining the Vulnerability in Language Models](https://mlsanity.com/paper/2305.01219/) (2023)
- [A Unified Evaluation of Textual Backdoor Learning: Frameworks and Benchmarks](https://mlsanity.com/paper/2206.08514/) (2022)
- [Triggerless Backdoor Attack for NLP Tasks with Clean Labels](https://mlsanity.com/paper/2111.07970/) (2021)
- [BadPrompt: Backdoor Attacks on Continuous Prompts](https://mlsanity.com/paper/2211.14719/) (2022)
- [TrojanPuzzle: Covertly Poisoning Code-Suggestion Models](https://mlsanity.com/paper/2301.02344/) (2023)
- [Rickrolling the Artist: Injecting Backdoors into Text Encoders for Text-to-Image Synthesis](https://mlsanity.com/paper/2211.02408/) (2022)
- [BITE: Textual Backdoor Attacks with Iterative Trigger Injection](https://mlsanity.com/paper/2205.12700/) (2022)
- [Semantic Mirror Jailbreak: Genetic Algorithm Based Jailbreak Prompts Against Open-source LLMs](https://mlsanity.com/paper/2402.14872/) (2024)
- [Stealthy Backdoor Attack for Code Models](https://mlsanity.com/paper/2301.02496/) (2023)
- [Constrained Optimization with Dynamic Bound-scaling for Effective NLPBackdoor Defense](https://mlsanity.com/paper/2202.05749/) (2022)
- [Operationalizing a Threat Model for Red-Teaming Large Language Models (LLMs)](https://mlsanity.com/paper/2407.14937/) (2024)
- [An Overview of Backdoor Attacks Against Deep Neural Networks and Possible Defences](https://mlsanity.com/paper/2111.08429/) (2021)
- [Backdoor Attacks and Countermeasures in Natural Language Processing Models: A Comprehensive Security Review](https://mlsanity.com/paper/2309.06055/) (2023)
- [NOTABLE: Transferable Backdoor Attacks Against Prompt-based NLP Models](https://mlsanity.com/paper/2305.17826/) (2023)
- [Attention-Enhancing Backdoor Attacks Against BERT-based Models](https://mlsanity.com/paper/2310.14480/) (2023)
- [ChatGPT as an Attack Tool: Stealthy Textual Backdoor Attack via Blackbox Generative Model Trigger](https://mlsanity.com/paper/2304.14475/) (2023)
- [TrojLLM: A Black-box Trojan Prompt Attack on Large Language Models](https://mlsanity.com/paper/2306.06815/) (2023)
- [Elijah: Eliminating Backdoors Injected in Diffusion Models via Distribution Shift](https://mlsanity.com/paper/2312.00050/) (2023)
- [Watermarking Text Data on Large Language Models for Dataset Copyright](https://mlsanity.com/paper/2305.13257/) (2023)
- [From Shortcuts to Triggers: Backdoor Defense with Denoised PoE](https://mlsanity.com/paper/2305.14910/) (2023)
- [Here's a Free Lunch: Sanitizing Backdoored Models with Model Merge](https://mlsanity.com/paper/2402.19334/) (2024)
- [Defending against Insertion-based Textual Backdoor Attacks via Attribution](https://mlsanity.com/paper/2305.02394/) (2023)
- [Kallima: A Clean-label Framework for Textual Backdoor Attacks](https://mlsanity.com/paper/2206.01832/) (2022)
- [Backdoored Retrievers for Prompt Injection Attacks on Retrieval Augmented Generation of Large Language Models](https://mlsanity.com/paper/2410.14479/) (2024)
- [Backdoor Learning on Sequence to Sequence Models](https://mlsanity.com/paper/2305.02424/) (2023)
- [Mitigating Backdoor Poisoning Attacks through the Lens of Spurious Correlation](https://mlsanity.com/paper/2305.11596/) (2023)
- [BagFlip: A Certified Defense against Data Poisoning](https://mlsanity.com/paper/2205.13634/) (2022)

## Abstract (arXiv)

Recent studies show that neural natural language processing (NLP) models are vulnerable to backdoor attacks. Injected with backdoors, models perform normally on benign examples but produce attacker-specified predictions when the backdoor is activated, presenting serious security threats to real-world applications. Since existing textual backdoor attacks pay little attention to the invisibility of backdoors, they can be easily detected and blocked. In this work, we present invisible backdoors that are activated by a learnable combination of word substitution. We show that NLP models can be injected with backdoors that lead to a nearly 100% attack success rate, whereas being highly invisible to existing defense strategies and even human inspections. The results raise a serious alarm to the security of NLP models, which requires further research to be resolved. All the data and code of this paper are released at https://github.com/thunlp/BkdAtk-LWS.