# Enhancing the Transferability of Adversarial Attacks through Variance Tuning

arXiv: https://arxiv.org/abs/2103.15571 · HTML: https://mlsanity.com/paper/2103.15571/

## Key facts

- Cited by 244 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 38 references that are in the map.
- First posted on arXiv 29 Mar 2021.

## Builds on (most cited first, 30 of 31)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Towards Evaluating the Robustness of Neural Networks](https://mlsanity.com/paper/1608.04644/) (2016)
- [Adversarial examples in the physical world](https://mlsanity.com/paper/1607.02533/) (2016)
- [Theoretically Principled Trade-off between Robustness and Accuracy](https://mlsanity.com/paper/1901.08573/) (2019)
- [Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning](https://mlsanity.com/paper/1602.07261/) (2016)
- [Certified Adversarial Robustness via Randomized Smoothing](https://mlsanity.com/paper/1902.02918/) (2019)
- [DeepFool: a simple and accurate method to fool deep neural networks](https://mlsanity.com/paper/1511.04599/) (2015)
- [Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples](https://mlsanity.com/paper/1802.00420/) (2018)
- [Adversarial Machine Learning at Scale](https://mlsanity.com/paper/1611.01236/) (2016)
- [Practical Black-Box Attacks against Machine Learning](https://mlsanity.com/paper/1602.02697/) (2016)
- [Ensemble Adversarial Training: Attacks and Defenses](https://mlsanity.com/paper/1705.07204/) (2017)
- [Improving Transferability of Adversarial Examples with Input Diversity](https://mlsanity.com/paper/1803.06978/) (2018)
- [Delving into Transferable Adversarial Examples and Black-box Attacks](https://mlsanity.com/paper/1611.02770/) (2016)
- [Countering Adversarial Images using Input Transformations](https://mlsanity.com/paper/1711.00117/) (2017)
- [Black-box Adversarial Attacks with Limited Queries and Information](https://mlsanity.com/paper/1804.08598/) (2018)
- [Feature Squeezing: Detecting Adversarial Examples in Deep Neural Networks](https://mlsanity.com/paper/1704.01155/) (2017)
- [Decision-Based Adversarial Attacks: Reliable Attacks Against Black-Box Machine Learning Models](https://mlsanity.com/paper/1712.04248/) (2017)
- [Evading Defenses to Transferable Adversarial Examples by Translation-Invariant Attacks](https://mlsanity.com/paper/1904.02884/) (2019)
- [Mitigating Adversarial Effects Through Randomization](https://mlsanity.com/paper/1711.01991/) (2017)
- [Nesterov Accelerated Gradient and Scale Invariance for Adversarial Attacks](https://mlsanity.com/paper/1908.06281/) (2019)
- [Defense against Adversarial Attacks Using High-Level Representation Guided Denoiser](https://mlsanity.com/paper/1712.02976/) (2017)
- [A Self-supervised Approach for Adversarial Robustness](https://mlsanity.com/paper/2006.04924/) (2020)
- [Admix: Enhancing the Transferability of Adversarial Attacks](https://mlsanity.com/paper/2102.00436/) (2021)
- [ComDefend: An Efficient Image Compression Model to Defend Adversarial Examples](https://mlsanity.com/paper/1811.12673/) (2018)
- [Feature Distillation: DNN-Oriented JPEG Compression Against Adversarial Examples](https://mlsanity.com/paper/1803.05787/) (2018)

## Built on by (most cited first, 30 of 244)

- [How Robust is Google's Bard to Adversarial Image Attacks?](https://mlsanity.com/paper/2309.11751/) (2023)
- [Frequency Domain Model Augmentation for Adversarial Attack](https://mlsanity.com/paper/2207.05382/) (2022)
- [Advances in adversarial attacks and defenses in computer vision: A survey](https://mlsanity.com/paper/2108.00401/) (2021)
- [LAS-AT: Adversarial Training with Learnable Attack Strategy](https://mlsanity.com/paper/2203.06616/) (2022)
- [Diffusion Models for Imperceptible and Transferable Adversarial Attack](https://mlsanity.com/paper/2305.08192/) (2023)
- [Rethinking Model Ensemble in Transfer-based Adversarial Attacks](https://mlsanity.com/paper/2303.09105/) (2023)
- [A Comprehensive Study on Robustness of Image Classification Models: Benchmarking and Rethinking](https://mlsanity.com/paper/2302.14301/) (2023)
- [Stochastic Variance Reduced Ensemble Adversarial Attack for Boosting the Adversarial Transferability](https://mlsanity.com/paper/2111.10752/) (2021)
- [A Survey of Attacks on Large Vision-Language Models: Resources, Advances, and Future Trends](https://mlsanity.com/paper/2407.07403/) (2024)
- [On Improving Adversarial Transferability of Vision Transformers](https://mlsanity.com/paper/2106.04169/) (2021)
- [Boosting the Transferability of Adversarial Attacks with Reverse Adversarial Perturbation](https://mlsanity.com/paper/2210.05968/) (2022)
- [Boosting Adversarial Transferability by Block Shuffle and Rotation](https://mlsanity.com/paper/2308.10299/) (2023)
- [Structure Invariant Transformation for better Adversarial Transferability](https://mlsanity.com/paper/2309.14700/) (2023)
- [A Large-scale Multiple-objective Method for Black-box Attack against Object Detection](https://mlsanity.com/paper/2209.07790/) (2022)
- [Improving the Transferability of Targeted Adversarial Examples through Object-Based Diverse Input](https://mlsanity.com/paper/2203.09123/) (2022)
- [Triangle Attack: A Query-efficient Decision-based Adversarial Attack](https://mlsanity.com/paper/2112.06569/) (2021)
- [Boosting Adversarial Transferability by Achieving Flat Local Maxima](https://mlsanity.com/paper/2306.05225/) (2023)
- [AnyAttack: Towards Large-scale Self-supervised Adversarial Attacks on Vision-language Models](https://mlsanity.com/paper/2410.05346/) (2024)
- [A Survey on Transferability of Adversarial Examples across Deep Neural Networks](https://mlsanity.com/paper/2310.17626/) (2023)
- [Adversarial Prompt Tuning for Vision-Language Models](https://mlsanity.com/paper/2311.11261/) (2023)
- [Adv-Attribute: Inconspicuous and Transferable Adversarial Attack on Face Recognition](https://mlsanity.com/paper/2210.06871/) (2022)
- [Transferable Adversarial Attacks on Vision Transformers with Token Gradient Regularization](https://mlsanity.com/paper/2303.15754/) (2023)
- [Boosting Fast Adversarial Training with Learnable Adversarial Initialization](https://mlsanity.com/paper/2110.05007/) (2021)
- [SA-Attack: Improving Adversarial Transferability of Vision-Language Pre-training Models via Self-Augmentation](https://mlsanity.com/paper/2312.04913/) (2023)
- [Evading DeepFake Detectors via Adversarial Statistical Consistency](https://mlsanity.com/paper/2304.11670/) (2023)
- [Improving the Transferability of Adversarial Samples by Path-Augmented Method](https://mlsanity.com/paper/2303.15735/) (2023)
- [Transferable Adversarial Attack based on Integrated Gradients](https://mlsanity.com/paper/2205.13152/) (2022)
- [Prior-Guided Adversarial Initialization for Fast Adversarial Training](https://mlsanity.com/paper/2207.08859/) (2022)
- [DISCO: Adversarial Defense with Local Implicit Functions](https://mlsanity.com/paper/2212.05630/) (2022)
- [Sibling-Attack: Rethinking Transferable Adversarial Attacks against Face Recognition](https://mlsanity.com/paper/2303.12512/) (2023)

## Abstract (arXiv)

Deep neural networks are vulnerable to adversarial examples that mislead the models with imperceptible perturbations. Though adversarial attacks have achieved incredible success rates in the white-box setting, most existing adversaries often exhibit weak transferability in the black-box setting, especially under the scenario of attacking models with defense mechanisms. In this work, we propose a new method called variance tuning to enhance the class of iterative gradient based attack methods and improve their attack transferability. Specifically, at each iteration for the gradient calculation, instead of directly using the current gradient for the momentum accumulation, we further consider the gradient variance of the previous iteration to tune the current gradient so as to stabilize the update direction and escape from poor local optima. Empirical results on the standard ImageNet dataset demonstrate that our method could significantly improve the transferability of gradient-based adversarial attacks. Besides, our method could be used to attack ensemble models or be integrated with various input transformations. Incorporating variance tuning with input transformations on iterative gradient-based attacks in the multi-model setting, the integrated method could achieve an average success rate of 90.1% against nine advanced defense methods, improving the current best attack performance significantly by 85.1% . Code is available at https://github.com/JHL-HUST/VT.