# Structure Invariant Transformation for better Adversarial Transferability

arXiv: https://arxiv.org/abs/2309.14700 · HTML: https://mlsanity.com/paper/2309.14700/

## Key facts

- Cited by 62 papers in the map (data through 14 Oct 2026).
- Builds on 65 of its 79 references that are in the map.
- First posted on arXiv 26 Sep 2023.

## Builds on (most cited first, 30 of 65)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features](https://mlsanity.com/paper/1905.04899/) (2019)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [Aggregated Residual Transformations for Deep Neural Networks](https://mlsanity.com/paper/1611.05431/) (2016)
- [Adversarial examples in the physical world](https://mlsanity.com/paper/1607.02533/) (2016)
- [Certified Adversarial Robustness via Randomized Smoothing](https://mlsanity.com/paper/1902.02918/) (2019)
- [DeepFool: a simple and accurate method to fool deep neural networks](https://mlsanity.com/paper/1511.04599/) (2015)
- [Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples](https://mlsanity.com/paper/1802.00420/) (2018)
- [CosFace: Large Margin Cosine Loss for Deep Face Recognition](https://mlsanity.com/paper/1801.09414/) (2018)
- [Ensemble Adversarial Training: Attacks and Defenses](https://mlsanity.com/paper/1705.07204/) (2017)
- [Fast is better than free: Revisiting adversarial training](https://mlsanity.com/paper/2001.03994/) (2020)
- [Square Attack: a query-efficient black-box adversarial attack via random search](https://mlsanity.com/paper/1912.00049/) (2019)
- [Improving Transferability of Adversarial Examples with Input Diversity](https://mlsanity.com/paper/1803.06978/) (2018)
- [Delving into Transferable Adversarial Examples and Black-box Attacks](https://mlsanity.com/paper/1611.02770/) (2016)
- [Countering Adversarial Images using Input Transformations](https://mlsanity.com/paper/1711.00117/) (2017)
- [Black-box Adversarial Attacks with Limited Queries and Information](https://mlsanity.com/paper/1804.08598/) (2018)
- [Feature Squeezing: Detecting Adversarial Examples in Deep Neural Networks](https://mlsanity.com/paper/1704.01155/) (2017)
- [Decision-Based Adversarial Attacks: Reliable Attacks Against Black-Box Machine Learning Models](https://mlsanity.com/paper/1712.04248/) (2017)

## Built on by (most cited first, 30 of 62)

- [SA-Attack: Improving Adversarial Transferability of Vision-Language Pre-training Models via Self-Augmentation](https://mlsanity.com/paper/2312.04913/) (2023)
- [Efficient Generation of Targeted and Transferable Adversarial Examples for Vision-Language Models Via Diffusion Models](https://mlsanity.com/paper/2404.10335/) (2024)
- [Learning to Transform Dynamically for Better Adversarial Transferability](https://mlsanity.com/paper/2405.14077/) (2024)
- [Attention! Your Vision Language Model Could Be Maliciously Manipulated](https://mlsanity.com/paper/2505.19911/) (2025)
- [Rethinking Impersonation and Dodging Attacks on Face Recognition Systems](https://mlsanity.com/paper/2401.08903/) (2024)
- [Boosting the Transferability of Adversarial Examples via Local Mixup and Adaptive Step Size](https://mlsanity.com/paper/2401.13205/) (2024)
- [Understanding Model Ensemble in Transferable Adversarial Attack](https://mlsanity.com/paper/2410.06851/) (2024)
- [Improving Adversarial Transferability on Vision Transformers via Forward Propagation Refinement](https://mlsanity.com/paper/2503.15404/) (2025)
- [SoK: Pitfalls in Evaluating Black-Box Attacks](https://mlsanity.com/paper/2310.17534/) (2023)
- [Adversarial Example Soups: Improving Transferability and Stealthiness for Free](https://mlsanity.com/paper/2402.18370/) (2024)
- [Benchmarking Transferable Adversarial Attacks](https://mlsanity.com/paper/2402.00418/) (2024)
- [Transfer Attack for Bad and Good: Explain and Boost Adversarial Transferability across Multimodal Large Language Models](https://mlsanity.com/paper/2405.20090/) (2024)
- [Boost Adversarial Transferability by Uniform Scale and Mix Mask Method](https://mlsanity.com/paper/2311.12051/) (2023)
- [Adversarial Attacks on Both Face Recognition and Face Anti-spoofing Models](https://mlsanity.com/paper/2405.16940/) (2024)
- [Improving the Transferability of Adversarial Attacks on Face Recognition with Diverse Parameters Augmentation](https://mlsanity.com/paper/2411.15555/) (2024)
- [Harnessing the Computation Redundancy in ViTs to Boost Adversarial Transferability](https://mlsanity.com/paper/2504.10804/) (2025)
- [Rethinking Mixup for Improving the Adversarial Transferability](https://mlsanity.com/paper/2311.17087/) (2023)
- [Exploring Frequencies via Feature Mixing and Meta-Learning for Improving Adversarial Transferability](https://mlsanity.com/paper/2405.03193/) (2024)
- [Any Target Can be Offense: Adversarial Example Generation via Generalized Latent Infection](https://mlsanity.com/paper/2407.12292/) (2024)
- [Data-free Universal Adversarial Perturbation with Pseudo-semantic Prior](https://mlsanity.com/paper/2502.21048/) (2025)
- [Boosting Adversarial Transferability via High-Frequency Augmentation and Hierarchical-Gradient Fusion](https://mlsanity.com/paper/2505.21181/) (2025)
- [Enhancing Diffusion-based Unrestricted Adversarial Attacks via Adversary Preferences Alignment](https://mlsanity.com/paper/2506.01511/) (2025)
- [ScoreAdv: Score-based Targeted Generation of Natural Adversarial Examples via Diffusion Models](https://mlsanity.com/paper/2507.06078/) (2025)
- [Forward Learning for Gradient-based Black-box Saliency Map Generation](https://mlsanity.com/paper/2403.15603/) (2024)
- [MAA: Meticulous Adversarial Attack against Vision-Language Pre-trained Models](https://mlsanity.com/paper/2502.08079/) (2025)
- [Boosting the Local Invariance for Better Adversarial Transferability](https://mlsanity.com/paper/2503.06140/) (2025)
- [DMS: Addressing Information Loss with More Steps for Pragmatic Adversarial Attacks](https://mlsanity.com/paper/2406.07580/) (2024)
- [On the Adversarial Transferability of Generalized "Skip Connections"](https://mlsanity.com/paper/2410.08950/) (2024)
- [S4ST: A Strong, Self-transferable, faSt, and Simple Scale Transformation for Transferable Targeted Attack](https://mlsanity.com/paper/2410.13891/) (2024)
- [Boosting Adversarial Transferability with Spatial Adversarial Alignment](https://mlsanity.com/paper/2501.01015/) (2025)

## Abstract (arXiv)

Given the severe vulnerability of Deep Neural Networks (DNNs) against adversarial examples, there is an urgent need for an effective adversarial attack to identify the deficiencies of DNNs in security-sensitive applications. As one of the prevalent black-box adversarial attacks, the existing transfer-based attacks still cannot achieve comparable performance with the white-box attacks. Among these, input transformation based attacks have shown remarkable effectiveness in boosting transferability. In this work, we find that the existing input transformation based attacks transform the input image globally, resulting in limited diversity of the transformed images. We postulate that the more diverse transformed images result in better transferability. Thus, we investigate how to locally apply various transformations onto the input image to improve such diversity while preserving the structure of image. To this end, we propose a novel input transformation based attack, called Structure Invariant Attack (SIA), which applies a random image transformation onto each image block to craft a set of diverse images for gradient calculation. Extensive experiments on the standard ImageNet dataset demonstrate that SIA exhibits much better transferability than the existing SOTA input transformation based attacks on CNN-based and transformer-based models, showing its generality and superiority in boosting transferability. Code is available at https://github.com/xiaosen-wang/SIT.