# Feature Importance-aware Transferable Adversarial Attacks

arXiv: https://arxiv.org/abs/2107.14185 · HTML: https://mlsanity.com/paper/2107.14185/

## Key facts

- Cited by 127 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 36 references that are in the map.
- First posted on arXiv 29 Jul 2021.

## Builds on (most cited first, 30 of 31)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [Towards Evaluating the Robustness of Neural Networks](https://mlsanity.com/paper/1608.04644/) (2016)
- [Fast R-CNN](https://mlsanity.com/paper/1504.08083/) (2015)
- [Identity Mappings in Deep Residual Networks](https://mlsanity.com/paper/1603.05027/) (2016)
- [Adversarial examples in the physical world](https://mlsanity.com/paper/1607.02533/) (2016)
- [Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning](https://mlsanity.com/paper/1602.07261/) (2016)
- [Adversarial Examples Are Not Bugs, They Are Features](https://mlsanity.com/paper/1905.02175/) (2019)
- [Adversarial Machine Learning at Scale](https://mlsanity.com/paper/1611.01236/) (2016)
- [Ensemble Adversarial Training: Attacks and Defenses](https://mlsanity.com/paper/1705.07204/) (2017)
- [Improving Transferability of Adversarial Examples with Input Diversity](https://mlsanity.com/paper/1803.06978/) (2018)
- [Delving into Transferable Adversarial Examples and Black-box Attacks](https://mlsanity.com/paper/1611.02770/) (2016)
- [Black-box Adversarial Attacks with Limited Queries and Information](https://mlsanity.com/paper/1804.08598/) (2018)
- [Decision-Based Adversarial Attacks: Reliable Attacks Against Black-Box Machine Learning Models](https://mlsanity.com/paper/1712.04248/) (2017)
- [Evading Defenses to Transferable Adversarial Examples by Translation-Invariant Attacks](https://mlsanity.com/paper/1904.02884/) (2019)
- [Nesterov Accelerated Gradient and Scale Invariance for Adversarial Attacks](https://mlsanity.com/paper/1908.06281/) (2019)
- [Generating Adversarial Examples with Adversarial Networks](https://mlsanity.com/paper/1801.02610/) (2018)
- [Adversarial Risk and the Dangers of Evaluating Against Weak Attacks](https://mlsanity.com/paper/1802.05666/) (2018)
- [Enhancing Adversarial Example Transferability with an Intermediate Level Attack](https://mlsanity.com/paper/1907.10823/) (2019)
- [Patch-wise Attack for Fooling Deep Neural Network](https://mlsanity.com/paper/2007.06765/) (2020)
- [Deeply learned face representations are sparse, selective, and robust](https://mlsanity.com/paper/1412.1265/) (2014)
- [FDA: Feature Disruptive Attack](https://mlsanity.com/paper/1909.04385/) (2019)
- [Transferable Perturbations of Deep Feature Distributions](https://mlsanity.com/paper/2004.12519/) (2020)
- [Enhancing Cross-task Black-Box Transferability of Adversarial Examples with Dispersion Reduction](https://mlsanity.com/paper/1911.11616/) (2019)

## Built on by (most cited first, 30 of 127)

- [Frequency Domain Model Augmentation for Adversarial Attack](https://mlsanity.com/paper/2207.05382/) (2022)
- [Decorate the Newcomers: Visual Domain Prompt for Continual Test Time Adaptation](https://mlsanity.com/paper/2212.04145/) (2022)
- [Improving Adversarial Transferability via Neuron Attribution-Based Attacks](https://mlsanity.com/paper/2204.00008/) (2022)
- [How Deep Learning Sees the World: A Survey on Adversarial Attacks & Defenses](https://mlsanity.com/paper/2305.10862/) (2023)
- [Boosting Adversarial Transferability by Block Shuffle and Rotation](https://mlsanity.com/paper/2308.10299/) (2023)
- [R.A.C.E.: Robust Adversarial Concept Erasure for Secure Text-to-Image Diffusion Model](https://mlsanity.com/paper/2405.16341/) (2024)
- [A Survey on Transferability of Adversarial Examples across Deep Neural Networks](https://mlsanity.com/paper/2310.17626/) (2023)
- [An Adaptive Model Ensemble Adversarial Attack for Boosting Adversarial Transferability](https://mlsanity.com/paper/2308.02897/) (2023)
- [Natural Color Fool: Towards Boosting Black-box Unrestricted Attacks](https://mlsanity.com/paper/2210.02041/) (2022)
- [Improving the Transferability of Adversarial Samples by Path-Augmented Method](https://mlsanity.com/paper/2303.15735/) (2023)
- [Diversifying the High-level Features for better Adversarial Transferability](https://mlsanity.com/paper/2304.10136/) (2023)
- [Enhancing the Self-Universality for Transferable Targeted Attacks](https://mlsanity.com/paper/2209.03716/) (2022)
- [Improving Adversarial Transferability via Intermediate-level Perturbation Decay](https://mlsanity.com/paper/2304.13410/) (2023)
- [Rethinking the Backward Propagation for Adversarial Transferability](https://mlsanity.com/paper/2306.12685/) (2023)
- [Adversarial Attacks of Vision Tasks in the Past 10 Years: A Survey](https://mlsanity.com/paper/2410.23687/) (2024)
- [Attacks in Adversarial Machine Learning: A Systematic Survey from the Life-cycle Perspective](https://mlsanity.com/paper/2302.09457/) (2023)
- [A Survey of Robustness and Safety of 2D and 3D Deep Learning Models Against Adversarial Attacks](https://mlsanity.com/paper/2310.00633/) (2023)
- [Boosting Adversarial Transferability across Model Genus by Deformation-Constrained Warping](https://mlsanity.com/paper/2402.03951/) (2024)
- [Improved Generation of Adversarial Examples Against Safety-aligned LLMs](https://mlsanity.com/paper/2405.20778/) (2024)
- [Introducing Competition to Boost the Transferability of Targeted Adversarial Examples through Clean Feature Mixup](https://mlsanity.com/paper/2305.14846/) (2023)
- [Exploring Transferability of Multimodal Adversarial Samples for Vision-Language Pre-training Models with Contrastive Learning](https://mlsanity.com/paper/2308.12636/) (2023)
- [Training Meta-Surrogate Model for Transferable Adversarial Attack](https://mlsanity.com/paper/2109.01983/) (2021)
- [Strong Transferable Adversarial Attacks via Ensembled Asymptotically Normal Distribution Learning](https://mlsanity.com/paper/2209.11964/) (2022)
- [Towards Good Practices in Evaluating Transfer Adversarial Attacks](https://mlsanity.com/paper/2211.09565/) (2022)
- [Practical Deep Dispersed Watermarking with Synchronization and Fusion](https://mlsanity.com/paper/2310.14532/) (2023)
- [Reliable Evaluation of Adversarial Transferability](https://mlsanity.com/paper/2306.08565/) (2023)
- [When Robots Obey the Patch: Universal Transferable Patch Attacks on Vision-Language-Action Models](https://mlsanity.com/paper/2511.21192/) (2025)
- [As Firm As Their Foundations: Can open-sourced foundation models be used to create adversarial examples for downstream tasks?](https://mlsanity.com/paper/2403.12693/) (2024)
- [FIT-Print: Towards False-claim-resistant Model Ownership Verification via Targeted Fingerprint](https://mlsanity.com/paper/2501.15509/) (2025)
- [Towards Model Resistant to Transferable Adversarial Examples via Trigger Activation](https://mlsanity.com/paper/2504.14541/) (2025)

## Abstract (arXiv)

Transferability of adversarial examples is of central importance for attacking an unknown model, which facilitates adversarial attacks in more practical scenarios, e.g., black-box attacks. Existing transferable attacks tend to craft adversarial examples by indiscriminately distorting features to degrade prediction accuracy in a source model without aware of intrinsic features of objects in the images. We argue that such brute-force degradation would introduce model-specific local optimum into adversarial examples, thus limiting the transferability. By contrast, we propose the Feature Importance-aware Attack (FIA), which disrupts important object-aware features that dominate model decisions consistently. More specifically, we obtain feature importance by introducing the aggregate gradient, which averages the gradients with respect to feature maps of the source model, computed on a batch of random transforms of the original clean image. The gradients will be highly correlated to objects of interest, and such correlation presents invariance across different models. Besides, the random transforms will preserve intrinsic features of objects and suppress model-specific information. Finally, the feature importance guides to search for adversarial examples towards disrupting critical features, achieving stronger transferability. Extensive experimental evaluation demonstrates the effectiveness and superior performance of the proposed FIA, i.e., improving the success rate by 9.5% against normally trained models and 12.8% against defense models as compared to the state-of-the-art transferable attacks. Code is available at: https://github.com/hcguoO0/FIA