mlsanityData to 14 Oct 2026

IMBERT: Making BERT Immune to Insertion-based Backdoor Attacks

arXiv 2305.16503 ↗First posted 25 May 2023cs.CLcited by 15

On the map: LLM safety and security → Backdoor and data poisoning attacks

Abstract (arXiv)

Backdoor attacks are an insidious security threat against machine learning models. Adversaries can manipulate the predictions of compromised models by inserting triggers into the training phase. Various backdoor attacks have been devised which can achieve nearly perfect attack success without affecting model predictions for clean inputs. Means of mitigating such vulnerabilities are underdeveloped, especially in natural language processing. To fill this gap, we introduce IMBERT, which uses either gradients or self-attention scores derived from victim models to self-defend against backdoor attacks at inference time. Our empirical studies demonstrate that IMBERT can effectively identify up to 98.5% of inserted triggers. Thus, it significantly reduces the attack success rate while attaining competitive accuracy on the clean dataset across widespread insertion-based attacks compared to two baselines. Finally, we show that our approach is model-agnostic, and can be easily ported to several pre-trained transformer models.

Key facts. Cited by 15 papers in the map (data through 14 Oct 2026). Placed in Backdoor and data poisoning attacks, one of 12 directions in LLM safety and security. Builds on 31 of its 48 references that are in the map. First posted on arXiv 25 May 2023.

Builds on

31 of 48 references.

Closest ancestors · 4

Same direction, newest first

  1. 2021BadPre: Task-agnostic Backdoor Attacks to Pre-trained NLP Foundation Models6 Oct
  2. 2021Putting words into the system's mouth: A targeted attack on neural machine translation using monolingual data poisoning12 Jul
  3. 2021Turn the Combination Lock: Learnable Textual Backdoor Attacks via Word Substitution11 Jun
  4. 2021Hidden Killer: Invisible Textual Backdoor Attacks with Syntactic Trigger26 May

Other references · 27

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  4. 2020Language Models are Few-Shot Learnerscited by 32,419
  5. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  6. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,899
  7. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,371
  8. 2015Character-level Convolutional Networks for Text Classificationcited by 2,330
  9. 2017BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chaincited by 1,106
  10. 2017Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoningcited by 1,055
  11. 2019Attention is not not Explanationcited by 636
  12. 2019Is Attention Interpretable?cited by 389
15 more
  1. 2019Hidden Trigger Backdoor Attackscited by 371
  2. 2020Reflection Backdoor: A Natural Backdoor Attack on Deep Neural Networkscited by 349
  3. 2020Weight Poisoning Attacks on Pre-trained Modelscited by 341
  4. 2019Predicting the Type and Target of Offensive Posts in Social Mediacited by 271
  5. 2020ONION: A Simple and Effective Defense Against Textual Backdoor Attackscited by 263
  6. 2019A backdoor attack against LSTM-based text classification systemscited by 227
  7. 2020Clean-Label Backdoor Attacks on Video Recognition Modelscited by 169
  8. 2021Learned Token Pruning for Transformerscited by 139
  9. 2018Backdoor Embedding in Convolutional Neural Network Models via Invisible Perturbationcited by 134
  10. 2019AllenNLP Interpret: A Framework for Explaining Predictions of NLP Modelscited by 74
  11. 2019Attention Interpretability Across NLP Taskscited by 71
  12. 2018Backdooring Convolutional Neural Networks via Targeted Weight Perturbationscited by 69
  13. 2023Mitigating Backdoor Poisoning Attacks through the Lens of Spurious Correlationcited by 21
  14. 2021Magic Pyramid: Accelerating Inference with Early Exiting and Token Pruningcited by 16
  15. 2021A General Framework for Defending Against Backdoor Attacks via Influence Graphcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 3; 2024: 5; 2025: 6; 2026: 1.

15 papers.

Sort:

By area:

  1. 2023Use of LLMs for Illicit Purposes: Threats, Prevention Measures, and Vulnerabilitiescited by 82
  2. 2023Composite Backdoor Attacks Against Large Language Modelscited by 73
  3. 2023Backdoor Attacks and Countermeasures in Natural Language Processing Models: A Comprehensive Security Reviewcited by 39
  4. 2024Here's a Free Lunch: Sanitizing Backdoored Models with Model Mergecited by 26
  5. 2024BadActs: A Universal Backdoor Defense in the Activation Spacecited by 21
  6. 2024TuBA: Cross-Lingual Transferability of Backdoor Attacks in LLMs with Instruction Tuningcited by 14
  7. 2024Turning Generative Models Degenerate: The Power of Data Poisoning Attackscited by 9
  8. 2025Concept-Level Explainability for Auditing & Steering LLM Responsescited by 6
  9. 2024SEEP: Training Dynamics Grounds Latent Representation Search for Mitigating Backdoor Poisoning Attackscited by 3
  10. 2025Detecting Stealthy Backdoor Samples based on Intra-class Distance for Large Language Modelscited by 3
  11. 2025NLP Security and Ethics, in the Wildcited by 2
  12. 2025SLIP: Soft Label Mechanism and Key-Extraction-Guided CoT-based Defense Against Instruction Backdoor in APIscited by 1
3 more
  1. 2025SteganoBackdoor: Evading Data-Poisoning Defenses via Steganographic Backdoorscited by 1
  2. 2025Backdoor Samples Detection Based on Perturbation Discrepancy Consistency in Pre-trained Language Modelscited by 0
  3. 2026An Empirical Study of Output-to-Input Loops for Black-Box Backdoor Detection in Fine-Tuned Open-Weight LLMscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Injecting Bias into Text Classification Models using Backdoor Attacks
  2. 2024Large Language Models are Good Attackers: Efficient and Stealthy Textual Backdoor Attacks
  3. 2022BITE: Textual Backdoor Attacks with Iterative Trigger Injection
  4. 2021Backdoor Pre-trained Models Can Transfer to All
  5. 2023Attention-Enhancing Backdoor Attacks Against BERT-based Models
  6. 2024Imperio: Language-Guided Backdoor Attacks for Arbitrary Model Control
  7. 2023PoisonPrompt: Backdoor Attack on Prompt-based Large Language Models
  8. 2024Gracefully Filtering Backdoor Samples for Generative Large Language Models without Retraining