# Boosting Adversarial Transferability by Block Shuffle and Rotation

arXiv: https://arxiv.org/abs/2308.10299 · HTML: https://mlsanity.com/paper/2308.10299/

## Key facts

- Cited by 64 papers in the map (data through 14 Oct 2026).
- Builds on 47 of its 59 references that are in the map.
- First posted on arXiv 20 Aug 2023.

## Builds on (most cited first, 30 of 47)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Towards Evaluating the Robustness of Neural Networks](https://mlsanity.com/paper/1608.04644/) (2016)
- [Learning Deep Features for Discriminative Localization](https://mlsanity.com/paper/1512.04150/) (2015)
- [Adversarial examples in the physical world](https://mlsanity.com/paper/1607.02533/) (2016)
- [Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning](https://mlsanity.com/paper/1602.07261/) (2016)
- [Certified Adversarial Robustness via Randomized Smoothing](https://mlsanity.com/paper/1902.02918/) (2019)
- [SmoothGrad: removing noise by adding noise](https://mlsanity.com/paper/1706.03825/) (2017)
- [CosFace: Large Margin Cosine Loss for Deep Face Recognition](https://mlsanity.com/paper/1801.09414/) (2018)
- [Practical Black-Box Attacks against Machine Learning](https://mlsanity.com/paper/1602.02697/) (2016)
- [Ensemble Adversarial Training: Attacks and Defenses](https://mlsanity.com/paper/1705.07204/) (2017)
- [Improving Transferability of Adversarial Examples with Input Diversity](https://mlsanity.com/paper/1803.06978/) (2018)
- [Delving into Transferable Adversarial Examples and Black-box Attacks](https://mlsanity.com/paper/1611.02770/) (2016)
- [Countering Adversarial Images using Input Transformations](https://mlsanity.com/paper/1711.00117/) (2017)
- [Feature Squeezing: Detecting Adversarial Examples in Deep Neural Networks](https://mlsanity.com/paper/1704.01155/) (2017)
- [Evading Defenses to Transferable Adversarial Examples by Translation-Invariant Attacks](https://mlsanity.com/paper/1904.02884/) (2019)
- [Mitigating Adversarial Effects Through Randomization](https://mlsanity.com/paper/1711.01991/) (2017)
- [Nesterov Accelerated Gradient and Scale Invariance for Adversarial Attacks](https://mlsanity.com/paper/1908.06281/) (2019)
- [Defense against Adversarial Attacks Using High-Level Representation Guided Denoiser](https://mlsanity.com/paper/1712.02976/) (2017)
- [Simple Black-box Adversarial Attacks](https://mlsanity.com/paper/1905.07121/) (2019)

## Built on by (most cited first, 30 of 64)

- [Learning to Transform Dynamically for Better Adversarial Transferability](https://mlsanity.com/paper/2405.14077/) (2024)
- [Transferable Adversarial Attacks on SAM and Its Downstream Models](https://mlsanity.com/paper/2410.20197/) (2024)
- [When Robots Obey the Patch: Universal Transferable Patch Attacks on Vision-Language-Action Models](https://mlsanity.com/paper/2511.21192/) (2025)
- [Attention! Your Vision Language Model Could Be Maliciously Manipulated](https://mlsanity.com/paper/2505.19911/) (2025)
- [Towards Model Resistant to Transferable Adversarial Examples via Trigger Activation](https://mlsanity.com/paper/2504.14541/) (2025)
- [From Pretrain to Pain: Adversarial Vulnerability of Video Foundation Models Without Task Knowledge](https://mlsanity.com/paper/2511.07049/) (2025)
- [Rethinking Impersonation and Dodging Attacks on Face Recognition Systems](https://mlsanity.com/paper/2401.08903/) (2024)
- [Boosting Adversarial Transferability with Low-Cost Optimization via Maximin Expected Flatness](https://mlsanity.com/paper/2405.16181/) (2024)
- [TASAR: Transfer-based Attack on Skeletal Action Recognition](https://mlsanity.com/paper/2409.02483/) (2024)
- [Understanding Model Ensemble in Transferable Adversarial Attack](https://mlsanity.com/paper/2410.06851/) (2024)
- [Improving Adversarial Transferability on Vision Transformers via Forward Propagation Refinement](https://mlsanity.com/paper/2503.15404/) (2025)
- [ForensicsSAM: Toward Robust and Unified Image Forgery Detection and Localization Resisting to Adversarial Attack](https://mlsanity.com/paper/2508.07402/) (2025)
- [Transferability Bound Theory: Exploring Relationship between Adversarial Transferability and Flatness](https://mlsanity.com/paper/2311.06423/) (2023)
- [Enhancing Targeted Adversarial Attacks on Large Vision-Language Models via Intermediate Projector](https://mlsanity.com/paper/2508.13739/) (2025)
- [Adversarial Attacks on Both Face Recognition and Face Anti-spoofing Models](https://mlsanity.com/paper/2405.16940/) (2024)
- [Improving the Transferability of Adversarial Attacks on Face Recognition with Diverse Parameters Augmentation](https://mlsanity.com/paper/2411.15555/) (2024)
- [Rethinking Mixup for Improving the Adversarial Transferability](https://mlsanity.com/paper/2311.17087/) (2023)
- [Exploring Frequencies via Feature Mixing and Meta-Learning for Improving Adversarial Transferability](https://mlsanity.com/paper/2405.03193/) (2024)
- [Data-free Universal Adversarial Perturbation with Pseudo-semantic Prior](https://mlsanity.com/paper/2502.21048/) (2025)
- [Boosting Adversarial Transferability via High-Frequency Augmentation and Hierarchical-Gradient Fusion](https://mlsanity.com/paper/2505.21181/) (2025)
- [Enhancing Diffusion-based Unrestricted Adversarial Attacks via Adversary Preferences Alignment](https://mlsanity.com/paper/2506.01511/) (2025)
- [ScoreAdv: Score-based Targeted Generation of Natural Adversarial Examples via Diffusion Models](https://mlsanity.com/paper/2507.06078/) (2025)
- [MAA: Meticulous Adversarial Attack against Vision-Language Pre-trained Models](https://mlsanity.com/paper/2502.08079/) (2025)
- [Information-Geometric Inverse Distillation for Enhancing Adversarial Transferability](https://mlsanity.com/paper/2502.17003/) (2025)
- [Boosting the Local Invariance for Better Adversarial Transferability](https://mlsanity.com/paper/2503.06140/) (2025)
- [GSBAK: top-K Geometric Score-based Black-box Attack](https://mlsanity.com/paper/2503.12827/) (2025)
- [Boosting Adversarial Transferability via Ensemble Non-Attention](https://mlsanity.com/paper/2511.08937/) (2025)
- [Generating Visually Realistic Adversarial Patch](https://mlsanity.com/paper/2312.03030/) (2023)
- [On the Adversarial Transferability of Generalized "Skip Connections"](https://mlsanity.com/paper/2410.08950/) (2024)
- [S4ST: A Strong, Self-transferable, faSt, and Simple Scale Transformation for Transferable Targeted Attack](https://mlsanity.com/paper/2410.13891/) (2024)

## Abstract (arXiv)

Adversarial examples mislead deep neural networks with imperceptible perturbations and have brought significant threats to deep learning. An important aspect is their transferability, which refers to their ability to deceive other models, thus enabling attacks in the black-box setting. Though various methods have been proposed to boost transferability, the performance still falls short compared with white-box attacks. In this work, we observe that existing input transformation based attacks, one of the mainstream transfer-based attacks, result in different attention heatmaps on various models, which might limit the transferability. We also find that breaking the intrinsic relation of the image can disrupt the attention heatmap of the original image. Based on this finding, we propose a novel input transformation based attack called block shuffle and rotation (BSR). Specifically, BSR splits the input image into several blocks, then randomly shuffles and rotates these blocks to construct a set of new images for gradient calculation. Empirical evaluations on the ImageNet dataset demonstrate that BSR could achieve significantly better transferability than the existing input transformation based methods under single-model and ensemble-model settings. Combining BSR with the current input transformation method can further improve the transferability, which significantly outperforms the state-of-the-art methods. Code is available at https://github.com/Trustworthy-AI-Group/BSR