mlsanityData to 7 Oct 2026

Expose Backdoors on the Way: A Feature-Based Efficient Defense against Textual Backdoor Attacks

arXiv 2210.07907 ↗First posted 14 Oct 2022cs.CL cs.AIcited by 18

On the map: LLM safety and security → Backdoor and data poisoning attacks

Abstract (arXiv)

Natural language processing (NLP) models are known to be vulnerable to backdoor attacks, which poses a newly arisen threat to NLP models. Prior online backdoor defense methods for NLP models only focus on the anomalies at either the input or output level, still suffering from fragility to adaptive attacks and high computational cost. In this work, we take the first step to investigate the unconcealment of textual poisoned samples at the intermediate-feature level and propose a feature-based efficient online defense method. Through extensive experiments on existing attacking methods, we find that the poisoned samples are far away from clean samples in the intermediate feature space of a poisoned NLP model. Motivated by this observation, we devise a distance-based anomaly score (DAN) to distinguish poisoned samples from clean samples at the feature level. Experiments on sentiment analysis and offense detection tasks demonstrate the superiority of DAN, as it substantially surpasses existing online defense methods in terms of defending performance and enjoys lower inference costs. Moreover, we show that DAN is also resistant to adaptive attacks based on feature-level regularization. Our code is available at https://github.com/lancopku/DAN.

Key facts. Cited by 18 papers in the map (data through 7 Oct 2026). Placed in Backdoor and data poisoning attacks, one of 12 directions in LLM safety and security. Builds on 44 of its 71 references that are in the map. First posted on arXiv 14 Oct 2022.

Builds on

44 of 71 references.

Closest ancestors · 7

Same direction, newest first

  1. 2022Backdoor Attack against NLP models with Robustness-Aware Perturbation defense8 Apr
  2. 2021RAP: Robustness-Aware Perturbations for Defending against Backdoor Attacks on NLP Models15 Oct
  3. 2021Mind the Style of Text! Adversarial and Backdoor Attacks Based on Text Style Transfer14 Oct
  4. 2021BadPre: Task-agnostic Backdoor Attacks to Pre-trained NLP Foundation Models6 Oct
  5. 2021Hidden Killer: Invisible Textual Backdoor Attacks with Syntactic Trigger26 May
  6. 2021Be Careful about Poisoned Word Embeddings: Exploring the Vulnerability of the Embedding Layers in NLP Models29 Mar
  7. 2021T-Miner: A Generative Approach to Defend Against Trojan Attacks on DNN-based Text Classification7 Mar

Other references · 37

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  5. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,841
  6. 2015Character-level Convolutional Networks for Text Classificationcited by 2,293
  7. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,157
  8. 2018A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attackscited by 1,472
  9. 2021DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharingcited by 1,219
  10. 2017BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chaincited by 1,052
  11. 2017Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoningcited by 1,012
  12. 2018Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networkscited by 550
25 more
  1. 2020Pre-trained Models for Natural Language Processing: A Surveycited by 504
  2. 2018Spectral Signatures in Backdoor Attackscited by 457
  3. 2019STRIP: A Defence Against Trojan Attacks on Deep Neural Networkscited by 437
  4. 2018Detecting Backdoor Attacks on Deep Neural Networks by Activation Clusteringcited by 429
  5. 2020Weight Poisoning Attacks on Pre-trained Modelscited by 318
  6. 2021Neural Attention Distillation: Erasing Backdoor Triggers from Deep Neural Networkscited by 313
  7. 2020Input-Aware Dynamic Backdoor Attackcited by 304
  8. 2018Characterizing Adversarial Subspaces Using Local Intrinsic Dimensionalitycited by 280
  9. 2018Large Scale Crowdsourcing and Characterization of Twitter Abusive Behaviorcited by 265
  10. 2020ONION: A Simple and Effective Defense Against Textual Backdoor Attackscited by 231
  11. 2019A backdoor attack against LSTM-based text classification systemscited by 214
  12. 2019Detecting AI Trojans Using Meta Neural Analysiscited by 192
  13. 2018SentiNet: Detecting Localized Universal Attacks Against Deep Learning Systemscited by 148
  14. 2021Backdoor Attacks on Pre-trained Models by Layerwise Weight Poisoningcited by 103
  15. 2021Backdoor Scanning for Deep Neural Networks through K-Arm Optimizationcited by 96
  16. 2019Defending Neural Backdoors via Generative Distribution Modelingcited by 82
  17. 2020Mitigating backdoor attacks in LSTM-based Text Classification Systems by Backdoor Keyword Identificationcited by 80
  18. 2020Trojaning Language Models for Fun and Profitcited by 80
  19. 2021Revisiting Mahalanobis Distance for Transformer-Based Out-of-Domain Detectioncited by 67
  20. 2019Design and Evaluation of a Multi-Domain Trojan Detection Method on Deep Neural Networkscited by 66
  21. 2020Feature Space Singularity for Out-of-Distribution Detectioncited by 49
  22. 2022A Study of the Attention Abnormality in Trojaned BERTscited by 47
  23. 2021How to Inject Backdoors with Better Consistency: Logit Anchoring on Clean Datacited by 22
  24. 2020Can We Mitigate Backdoor Attack Using Adversarial Detection Methods?cited by 8
  25. 2022Rethinking Textual Adversarial Defense for Pre-trained Language Modelscited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 4; 2024: 8; 2025: 4; 2026: 2.

18 papers.

Sort:

By area:

  1. 2024Watch Out for Your Agents! Investigating Backdoor Threats to LLM-Based Agentscited by 135
  2. 2023Use of LLMs for Illicit Purposes: Threats, Prevention Measures, and Vulnerabilitiescited by 76
  3. 2023Defending against Insertion-based Textual Backdoor Attacks via Attributioncited by 24
  4. 2024Here's a Free Lunch: Sanitizing Backdoored Models with Model Mergecited by 21
  5. 2024Unlearning Backdoor Attacks for LLMs with Weak-to-Strong Knowledge Distillationcited by 21
  6. 2023Mitigating Backdoor Poisoning Attacks through the Lens of Spurious Correlationcited by 19
  7. 2024BadActs: A Universal Backdoor Defense in the Activation Spacecited by 19
  8. 2023Setting the Trap: Capturing and Defeating Backdoors in Pretrained Language Models through Honeypotscited by 16
  9. 2024Defending Against Weight-Poisoning Backdoor Attacks for Parameter-Efficient Fine-Tuningcited by 16
  10. 2025Probe before You Talk: Towards Black-box Defense against Backdoor Unalignment for Large Language Modelscited by 15
  11. 2024Rethinking Backdoor Detection Evaluation for Language Modelscited by 6
  12. 2024SEEP: Training Dynamics Grounds Latent Representation Search for Mitigating Backdoor Poisoning Attackscited by 3
6 more
  1. 2025NLP Security and Ethics, in the Wildcited by 2
  2. 2025DUP: Detection-guided Unlearning for Backdoor Purification in Language Modelscited by 2
  3. 2026Benchmarking and Improving Monitors for Out-Of-Distribution Alignment Failure in LLMscited by 2
  4. 2024Defense Against Syntactic Textual Backdoor Attacks with Token Substitutioncited by 1
  5. 2025CUBA: Controlled Untargeted Backdoor Attack against Deep Neural Networkscited by 1
  6. 2026When Emotion Becomes Trigger: Emotion-style dynamic Backdoor Attack Parasitising Large Language Modelscited by 0