mlsanityData to 14 Oct 2026

One Pixel is All I Need

arXiv 2412.10681 ↗First posted 14 Dec 2024cs.CVcited by 0

On the map: LLM safety and security → Backdoor and data poisoning attacks

Abstract (arXiv)

Vision Transformers (ViTs) have achieved record-breaking performance in various visual tasks. However, concerns about their robustness against backdoor attacks have grown. Backdoor attacks involve associating a specific trigger with a target label, causing the model to predict the attacker-specified label when the trigger is present, while correctly identifying clean images.We found that ViTs exhibit higher attack success rates for quasi-triggers(patterns different from but similar to the original training triggers)compared to CNNs. Moreover, some backdoor features in clean samples can suppress the original trigger, making quasi-triggers more effective.To better understand and exploit these vulnerabilities, we developed a tool called the Perturbation Sensitivity Distribution Map (PSDM). PSDM computes and sums gradients over many inputs to show how sensitive the model is to small changes in the input. In ViTs, PSDM reveals a patch-like pattern where central pixels are more sensitive than edges. We use PSDM to guide the creation of quasi-triggers.Based on these findings, we designed "WorstVIT," a simple yet effective data poisoning backdoor for ViT models. This attack requires an extremely low poisoning rate, trains for just one epoch, and modifies a single pixel to successfully attack all validation images.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Backdoor and data poisoning attacks, one of 12 directions in LLM safety and security. Builds on 38 of its 45 references that are in the map. First posted on arXiv 14 Dec 2024.

Builds on

38 of 45 references.

Closest ancestors · 1

Same direction, newest first

  1. 2021Targeted Attack against Deep Neural Networks via Flipping Limited Weight Bits21 Feb

Other references · 37

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  4. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  5. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  6. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,733
  7. 2013Intriguing properties of neural networkscited by 5,155
  8. 2020Training data-efficient image transformers & distillation through attentioncited by 4,586
  9. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,548
  10. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,790
  11. 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,962
  12. 2017SmoothGrad: removing noise by adding noisecited by 1,109
25 more
  1. 2017BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chaincited by 1,106
  2. 2021Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNetcited by 1,101
  3. 2017Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoningcited by 1,055
  4. 2021Segmenter: Transformer for Semantic Segmentationcited by 855
  5. 2021Transformer in Transformercited by 633
  6. 2017One pixel attack for fooling deep neural networkscited by 582
  7. 2018Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networkscited by 580
  8. 2021Early Convolutions Help Transformers See Bettercited by 396
  9. 2019Hidden Trigger Backdoor Attackscited by 371
  10. 2021Rethinking Spatial Dimensions of Vision Transformerscited by 361
  11. 2022EfficientFormer: Vision Transformers at MobileNet Speedcited by 241
  12. 2021On the Robustness of Vision Transformers to Adversarial Examplescited by 167
  13. 2021Visformer: The Vision-friendly Transformercited by 137
  14. 2023SCALE-UP: An Efficient Black-box Input-level Backdoor Detection via Analyzing Scaled Prediction Consistencycited by 97
  15. 2021FIBA: Frequency-Injection based Backdoor Attack in Medical Image Analysiscited by 61
  16. 2021Adversarial Robustness Comparison of Vision Transformer and MLP-Mixer to CNNscited by 57
  17. 2023Generative Pre-trained Transformer: A Comprehensive Review on Enabling Technologies, Potential Applications, Emerging Challenges, and Future Directionscited by 55
  18. 2021Reveal of Vision Transformers Robustness against Adversarial Attackscited by 43
  19. 2023Backdoor Attacks Against Deep Image Compression via Adaptive Frequency Triggercited by 40
  20. 2021Imperceptible Transfer Attack and Defense on 3D Point Cloud Classificationcited by 36
  21. 2023TransXNet: Learning Both Global and Local Dynamics with a Dual Dynamic Token Mixer for Visual Recognitioncited by 21
  22. 2024Nearest is Not Dearest: Towards Practical Defense against Quantization-conditioned Backdoor Attackscited by 18
  23. 2021Backdoor Attack on Hash-based Image Retrieval via Clean-label Data Poisoningcited by 12
  24. 2023Beating Backdoor Attack at Its Own Gamecited by 11
  25. 2022Backdoor Attacks on Crowd Countingcited by 10

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2022Defending Backdoor Attacks on Vision Transformer via Patch Processing
  2. 2022Backdoor Attacks on Vision Transformers
  3. 2026Backdoor Directions in Vision Transformers
  4. 2022TrojViT: Trojan Insertion in Vision Transformers
  5. 2026PASTA: A Patch-Agnostic Twofold-Stealthy Backdoor Attack on Vision Transformers
  6. 2025BadViM: Backdoor Attack against Vision Mamba
  7. 2024Backdoor Attack Against Vision Transformers via Attention Gradient-Based Image Erosion
  8. 2026DF-LoGiT: Data-Free Logic-Gated Backdoor Attacks in Vision Transformers