mlsanityData to 7 Oct 2026

Skip Connections Matter: On the Transferability of Adversarial Examples Generated with ResNets

arXiv 2002.05990 ↗First posted 14 Feb 2020cs.LG cs.CR cs.CVcited by 165

Not on the map. Why?

Abstract (arXiv)

Skip connections are an essential component of current state-of-the-art deep neural networks (DNNs) such as ResNet, WideResNet, DenseNet, and ResNeXt. Despite their huge success in building deeper and more powerful DNNs, we identify a surprising security weakness of skip connections in this paper. Use of skip connections allows easier generation of highly transferable adversarial examples. Specifically, in ResNet-like (with skip connections) neural networks, gradients can backpropagate through either skip connections or residual modules. We find that using more gradients from the skip connections rather than the residual modules according to a decay factor, allows one to craft adversarial examples with high transferability. Our method is termed Skip Gradient Method(SGM). We conduct comprehensive transfer attacks against state-of-the-art DNNs including ResNets, DenseNets, Inceptions, Inception-ResNet, Squeeze-and-Excitation Network (SENet) and robustly trained DNNs. We show that employing SGM on the gradient flow can greatly improve the transferability of crafted attacks in almost all cases. Furthermore, SGM can be easily combined with existing black-box attack techniques, and obtain high improvements over state-of-the-art transferability methods. Our findings not only motivate new research into the architectural vulnerability of DNNs, but also open up further challenges for the design of secure DNN architectures.

Key facts. Cited by 165 papers in the map (data through 7 Oct 2026). First posted on arXiv 14 Feb 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 40; 2022: 30; 2023: 40; 2024: 23; 2025: 25; 2026: 7.

165 papers.

Sort:

By area:

  1. 2021Neural Attention Distillation: Erasing Backdoor Triggers from Deep Neural Networkscited by 313
  2. 2021Unlearnable Examples: Making Personal Data Unexploitablecited by 188
  3. 2021Admix: Enhancing the Transferability of Adversarial Attackscited by 155
  4. 2022Frequency Domain Model Augmentation for Adversarial Attackcited by 128
  5. 2021Advances in adversarial attacks and defenses in computer vision: A surveycited by 127
  6. 2021Towards Transferable Adversarial Attacks on Vision Transformerscited by 80
  7. 2021RobustART: Benchmarking Robustness on Architecture Design and Training Techniquescited by 78
  8. 2021Improving Adversarial Robustness via Channel-wise Activation Suppressingcited by 76
  9. 2021On Improving Adversarial Transferability of Vision Transformerscited by 70
  10. 2022Boosting the Transferability of Adversarial Attacks with Reverse Adversarial Perturbationcited by 67
  11. 2023VLATTACK: Multimodal Adversarial Attacks on Vision-Language Tasks via Pre-trained Modelscited by 66
  12. 2021On Generating Transferable Targeted Perturbationscited by 65
38 more
  1. 2024Fight Back Against Jailbreaking via Prompt Adversarial Tuningcited by 63
  2. 2023Structure Invariant Transformation for better Adversarial Transferabilitycited by 61
  3. 2021Adversarial Robustness Comparison of Vision Transformer and MLP-Mixer to CNNscited by 57
  4. 2021Targeted Attack against Deep Neural Networks via Flipping Limited Weight Bitscited by 49
  5. 2023Boosting Adversarial Transferability by Achieving Flat Local Maximacited by 48
  6. 2023A Survey on Transferability of Adversarial Examples across Deep Neural Networkscited by 43
  7. 2022Beyond ImageNet Attack: Towards Crafting Adversarial Examples for Black-box Domainscited by 42
  8. 2022Enhancing Adversarial Training with Second-Order Statistics of Weightscited by 42
  9. 2023Transferable Adversarial Attacks on Vision Transformers with Token Gradient Regularizationcited by 42
  10. 2022LGV: Boosting Adversarial Example Transferability from Large Geometric Vicinitycited by 39
  11. 2023SA-Attack: Improving Adversarial Transferability of Vision-Language Pre-training Models via Self-Augmentationcited by 39
  12. 2022Towards Understanding and Boosting Adversarial Transferability from a Distribution Perspectivecited by 36
  13. 2022Transferable Adversarial Attack based on Integrated Gradientscited by 35
  14. 2023Diversifying the High-level Features for better Adversarial Transferabilitycited by 34
  15. 2021Analysis and Applications of Class-wise Robustness in Adversarial Trainingcited by 33
  16. 2021A Little Robustness Goes a Long Way: Leveraging Robust Features for Targeted Transfer Attackscited by 33
  17. 2022DISCO: Adversarial Defense with Local Implicit Functionscited by 33
  18. 2023Why Does Little Robustness Help? A Further Step Towards Understanding Adversarial Transferabilitycited by 32
  19. 2021Boosting Transferability of Targeted Adversarial Examples via Hierarchical Generative Networkscited by 31
  20. 2022Enhancing the Self-Universality for Transferable Targeted Attackscited by 30
  21. 2023Generalizable Black-Box Adversarial Attack with Meta Learningcited by 29
  22. 2023Rethinking the Backward Propagation for Adversarial Transferabilitycited by 27
  23. 2022Investigating Top-k White-Box and Transferable Black-box Attackcited by 26
  24. 2024Learning to Transform Dynamically for Better Adversarial Transferabilitycited by 23
  25. 2021Adaptive Image Transformations for Transfer-based Adversarial Attackcited by 22
  26. 2022OPOM: Customized Invisible Cloak towards Face Privacy Protectioncited by 22
  27. 2021Boosting the Transferability of Video Adversarial Examples via Temporal Translationcited by 21
  28. 2022Jigsaw-ViT: Learning Jigsaw Puzzles in Vision Transformercited by 21
  29. 2025X-Transfer Attacks: Towards Super Transferable Adversarial Attacks on CLIPcited by 21
  30. 2023CGBA: Curvature-aware Geometric Black-box Attackcited by 20
  31. 2024Transferable Adversarial Attacks on SAM and Its Downstream Modelscited by 20
  32. 2023Attacks in Adversarial Machine Learning: A Systematic Survey from the Life-cycle Perspectivecited by 19
  33. 2023Logit Margin Matters: Improving Transferable Targeted Adversarial Attack by Logit Calibrationcited by 18
  34. 2021A Unified Game-Theoretic Interpretation of Adversarial Robustnesscited by 17
  35. 2023Exploring Transferability of Multimodal Adversarial Samples for Vision-Language Pre-training Models with Contrastive Learningcited by 16
  36. 2023Adversarial Examples Are Not Real Featurescited by 16
  37. 2021Training Meta-Surrogate Model for Transferable Adversarial Attackcited by 15
  38. 2021Adversarial Examples on Segmentation Models Can be Easy to Transfercited by 15

50 most cited of 165 All 165 (JSON)