mlsanityData to 14 Oct 2026

Attention Hijacking in Trojan Transformers

arXiv 2208.04946 ↗First posted 9 Aug 2022cs.LG cs.AI cs.CRcited by 6

Not on the map. Why?

Abstract (arXiv)

Trojan attacks pose a severe threat to AI systems. Recent works on Transformer models received explosive popularity and the self-attentions are now indisputable. This raises a central question: Can we reveal the Trojans through attention mechanisms in BERTs and ViTs? In this paper, we investigate the attention hijacking pattern in Trojan AIs, ie, the trigger token ``kidnaps'' the attention weights when a specific trigger is present. We observe the consistent attention hijacking pattern in Trojan Transformers from both Natural Language Processing (NLP) and Computer Vision (CV) domains. This intriguing property helps us to understand the Trojan mechanism in BERTs and ViTs. We also propose an Attention-Hijacking Trojan Detector (AHTD) to discriminate the Trojan AIs from the clean ones.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 43 of its 60 references that are in the map. First posted on arXiv 9 Aug 2022.

Builds on

43 of 60 references.

References · 43

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  4. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  5. 2013Intriguing properties of neural networkscited by 5,155
  6. 2015Character-level Convolutional Networks for Text Classificationcited by 2,330
  7. 2019Similarity of Neural Network Representations Revisitedcited by 1,755
  8. 2017BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chaincited by 1,106
  9. 2017Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoningcited by 1,055
  10. 2019What Does BERT Look At? An Analysis of BERT's Attentioncited by 1,003
  11. 2019Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Prunedcited by 856
  12. 2019Are Sixteen Heads Really Better than One?cited by 816
31 more
  1. 2016Universal adversarial perturbationscited by 784
  2. 2019Universal Adversarial Triggers for Attacking and Analyzing NLPcited by 722
  3. 2021Do Vision Transformers See Like Convolutional Neural Networks?cited by 675
  4. 2020Backdoor Learning: A Surveycited by 426
  5. 2019Hidden Trigger Backdoor Attackscited by 371
  6. 2020Reflection Backdoor: A Natural Backdoor Attack on Deep Neural Networkscited by 349
  7. 2020Input-Aware Dynamic Backdoor Attackcited by 313
  8. 2022How Do Vision Transformers Work?cited by 307
  9. 2020ONION: A Simple and Effective Defense Against Textual Backdoor Attackscited by 263
  10. 2020Do Wide and Deep Networks Learn the Same Things? Uncovering How Neural Network Representations Vary with Width and Depthcited by 245
  11. 2019A backdoor attack against LSTM-based text classification systemscited by 227
  12. 2020Clean-Label Backdoor Attacks on Video Recognition Modelscited by 169
  13. 2019Universal Litmus Patterns: Revealing Backdoor Attacks in CNNscited by 146
  14. 2020Backdoor Attacks Against Deep Learning Systems in the Physical Worldcited by 134
  15. 2020Self-Attention Attribution: Interpreting Information Interactions Inside Transformercited by 125
  16. 2021Be Careful about Poisoned Word Embeddings: Exploring the Vulnerability of the Embedding Layers in NLP Modelscited by 105
  17. 2021RAP: Robustness-Aware Perturbations for Defending against Backdoor Attacks on NLP Modelscited by 104
  18. 2021Backdoor Scanning for Deep Neural Networks through K-Arm Optimizationcited by 98
  19. 2020Practical Detection of Trojan Neural Networks: Data-Limited and Data-Free Casescited by 95
  20. 2021T-Miner: A Generative Approach to Defend Against Trojan Attacks on DNN-based Text Classificationcited by 60
  21. 2020Can Adversarial Weight Perturbations Inject Neural Backdoors?cited by 56
  22. 2020Universal Adversarial Attacks with Natural Triggers for Text Classificationcited by 51
  23. 2022A Study of the Attention Abnormality in Trojaned BERTscited by 48
  24. 2021An Overview of Backdoor Attacks Against Deep Neural Networks and Possible Defencescited by 42
  25. 2021Trigger Hunting with a Topological Prior for Trojan Detectioncited by 39
  26. 2021Topological Detection of Trojaned Neural Networkscited by 34
  27. 2020Poison Attacks against Text Datasets with Conditional Adversarially Regularized Autoencodercited by 32
  28. 2020BAAAN: Backdoor Attacks Against Autoencoder and GAN-Based Machine Learning Modelscited by 25
  29. 2022A Survey of Neural Trojan Attacks and Defenses in Deep Learningcited by 17
  30. 2021On the Distribution, Sparsity, and Inference-time Quantization of Attention Values in Transformerscited by 14
  31. 2020Live Trojan Attacks on Deep Neural Networkscited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 4; 2025: 1; 2026: 1.

6 papers.

  1. 2024TrojVLM: Backdoor Attack Against Vision Language Modelscited by 39
  2. 2024Comprehensive evaluation of Mal-API-2019 dataset by machine learning in malware detectioncited by 8
  3. 2024BadCLM: Backdoor Attack in Clinical Language Models for Electronic Health Recordscited by 5
  4. 2024Towards SAR Automatic Target Recognition MultiCategory SAR Image Classification Based on Light Weight Vision Transformercited by 5
  5. 2025Backdoors in DRL: Four Environments Focusing on In-distribution Triggerscited by 2
  6. 2026Backdoor Learning in Language Models and Vision-Language Modelscited by 0