mlsanityData to 14 Oct 2026

Harnessing the Computation Redundancy in ViTs to Boost Adversarial Transferability

arXiv 2504.10804 ↗First posted 15 Apr 2025cs.CVcited by 4

Not on the map. Why?

Abstract (arXiv)

Vision Transformers (ViTs) have demonstrated impressive performance across a range of applications, including many safety-critical tasks. However, their unique architectural properties raise new challenges and opportunities in adversarial robustness. In particular, we observe that adversarial examples crafted on ViTs exhibit higher transferability compared to those crafted on CNNs, suggesting that ViTs contain structural characteristics favorable for transferable attacks. In this work, we investigate the role of computational redundancy in ViTs and its impact on adversarial transferability. Unlike prior studies that aim to reduce computation for efficiency, we propose to exploit this redundancy to improve the quality and transferability of adversarial examples. Through a detailed analysis, we identify two forms of redundancy, including the data-level and model-level, that can be harnessed to amplify attack effectiveness. Building on this insight, we design a suite of techniques, including attention sparsity manipulation, attention head permutation, clean token regularization, ghost MoE diversification, and test-time adversarial training. Extensive experiments on the ImageNet-1k dataset validate the effectiveness of our approach, showing that our methods significantly outperform existing baselines in both transferability and generality across diverse model architectures.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 56 of its 68 references that are in the map. First posted on arXiv 15 Apr 2025.

Builds on

56 of 68 references.

References · 56

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  4. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  5. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  6. 2025Qwen2.5-VL Technical Reportcited by 5,587
  7. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,475
  8. 2013Intriguing properties of neural networkscited by 5,155
  9. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,795
  10. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,998
  11. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
  12. 2015DeepFool: a simple and accurate method to fool deep neural networkscited by 1,318
38 more
  1. 2022Token Merging: Your ViT But Fastercited by 1,021
  2. 2019Square Attack: a query-efficient black-box adversarial attack via random searchcited by 685
  3. 2021Do Vision Transformers See Like Convolutional Neural Networks?cited by 675
  4. 2018Improving Transferability of Adversarial Examples with Input Diversitycited by 599
  5. 2016Delving into Transferable Adversarial Examples and Black-box Attackscited by 536
  6. 2021Intriguing Properties of Vision Transformerscited by 446
  7. 2019Evading Defenses to Transferable Adversarial Examples by Translation-Invariant Attackscited by 442
  8. 2024LLaVA-PruMerge: Adaptive Token Reduction for Efficient Large Multimodal Modelscited by 402
  9. 2021AdaViT: Adaptive Tokens for Efficient Vision Transformercited by 362
  10. 2021Rethinking Spatial Dimensions of Vision Transformerscited by 361
  11. 2019Nesterov Accelerated Gradient and Scale Invariance for Adversarial Attackscited by 355
  12. 2024MoE-LLaVA: Mixture of Experts for Large Vision-Language Modelscited by 300
  13. 2021Enhancing the Transferability of Adversarial Attacks through Variance Tuningcited by 244
  14. 2021Admix: Enhancing the Transferability of Adversarial Attackscited by 156
  15. 2022Frequency Domain Model Augmentation for Adversarial Attackcited by 130
  16. 2024Not All Heads Matter: A Head-Level KV Cache Compression Method with Integrated Retrieval and Reasoningcited by 97
  17. 2023Rethinking Model Ensemble in Transfer-based Adversarial Attackscited by 89
  18. 2021Stochastic Variance Reduced Ensemble Adversarial Attack for Boosting the Adversarial Transferabilitycited by 84
  19. 2024LLaVA-MoLE: Sparse Mixture of LoRA Experts for Mitigating Data Conflicts in Instruction Finetuning MLLMscited by 84
  20. 2024What Matters in Transformers? Not All Attention is Neededcited by 84
  21. 2020Patch-wise Attack for Fooling Deep Neural Networkcited by 78
  22. 2018Learning Transferable Adversarial Examples via Ghost Networkscited by 70
  23. 2022UIA-ViT: Unsupervised Inconsistency-Aware Method based on Vision Transformer for Face Forgery Detectioncited by 69
  24. 2023Structure Invariant Transformation for better Adversarial Transferabilitycited by 62
  25. 2021Boosting Adversarial Transferability through Enhanced Momentumcited by 54
  26. 2021Triangle Attack: A Query-efficient Decision-based Adversarial Attackcited by 48
  27. 2023Boosting Adversarial Transferability by Achieving Flat Local Maximacited by 48
  28. 2023Transferable Adversarial Attacks on Vision Transformers with Token Gradient Regularizationcited by 42
  29. 2022LGV: Boosting Adversarial Example Transferability from Large Geometric Vicinitycited by 39
  30. 2019advPattern: Physical-World Attacks on Deep Person Re-Identification via Adversarially Transformable Patternscited by 35
  31. 2020Adversarial Example Gamescited by 34
  32. 2023Diversifying the High-level Features for better Adversarial Transferabilitycited by 34
  33. 2023Making Substitute Models More Bayesian Can Enhance Transferability of Adversarial Examplescited by 32
  34. 2022Boosting the Transferability of Adversarial Attacks with Global Momentum Initializationcited by 29
  35. 2023Towards Transferable Targeted 3D Adversarial Attack in the Physical Worldcited by 25
  36. 2024Learning to Transform Dynamically for Better Adversarial Transferabilitycited by 23
  37. 2022On the Role of Generalization in Transferability of Adversarial Examplescited by 9
  38. 2023Improving the Transferability of Adversarial Examples via Direction Tuningcited by 9

50 most cited of 56 All 56 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 3.

4 papers.

  1. 2025Enhancing Adversarial Transferability through Block Stretch and Shrinkcited by 0
  2. 2026Devling into Adversarial Transferability on Image Classification: Review, Benchmark, and Evaluationcited by 0
  3. 2026Fast Preemptive Robustification: High-Frequency Response Anti-Aligns Shared Vulnerabilitycited by 0
  4. 2026Anchoring Adversarial Trajectories to Data Manifolds: A Bilevel Transfer Optimization Frameworkcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026A Survey of Adversarial Efficiency Degradation for Vision Transformer by Exploiting Input-adaptive Optimization
  2. 2026A Mechanistic Analysis of Adversarial Fine-tuning of Vision Transformers
  3. 2026MOAT: Model-Agnostic Randomized Transformations for preventing Efficiency Degradation Attacks on ViTs
  4. 2024Boosting Transferability in Vision-Language Attacks via Diversification along the Intersection Region of Adversarial Trajectory
  5. 2022Defending Backdoor Attacks on Vision Transformer via Patch Processing
  6. 2023Boosting Adversarial Transferability using Dynamic Cues
  7. 2026Improving Adversarial Transferability on Vision-Language Pre-training Models via Surrogate-Specific Bias Correction
  8. 2023MIMIR: Masked Image Modeling for Mutual Information-based Adversarial Robustness