# Closer Look at the Transferability of Adversarial Examples: How They Fool Different Models Differently

arXiv: https://arxiv.org/abs/2112.14337 · HTML: https://mlsanity.com/paper/2112.14337/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 34 references that are in the map.
- First posted on arXiv 29 Dec 2021.

## Builds on (most cited first, 24 of 24)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Towards Evaluating the Robustness of Neural Networks](https://mlsanity.com/paper/1608.04644/) (2016)
- [Shortcut Learning in Deep Neural Networks](https://mlsanity.com/paper/2004.07780/) (2020)
- [Adversarial examples in the physical world](https://mlsanity.com/paper/1607.02533/) (2016)
- [Fine-Grained Visual Classification of Aircraft](https://mlsanity.com/paper/1306.5151/) (2013)
- [DeepFool: a simple and accurate method to fool deep neural networks](https://mlsanity.com/paper/1511.04599/) (2015)
- [Adversarial Examples Are Not Bugs, They Are Features](https://mlsanity.com/paper/1905.02175/) (2019)
- [Practical Black-Box Attacks against Machine Learning](https://mlsanity.com/paper/1602.02697/) (2016)
- [Robustness May Be at Odds with Accuracy](https://mlsanity.com/paper/1805.12152/) (2018)
- [Improving Transferability of Adversarial Examples with Input Diversity](https://mlsanity.com/paper/1803.06978/) (2018)
- [Delving into Transferable Adversarial Examples and Black-box Attacks](https://mlsanity.com/paper/1611.02770/) (2016)
- [Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples](https://mlsanity.com/paper/1605.07277/) (2016)
- [The Space of Transferable Adversarial Examples](https://mlsanity.com/paper/1704.03453/) (2017)
- [Cross-Domain Transferability of Adversarial Perturbations](https://mlsanity.com/paper/1905.11736/) (2019)
- [Measuring the tendency of CNNs to Learn Surface Statistical Regularities](https://mlsanity.com/paper/1711.11561/) (2017)
- [On Generating Transferable Targeted Perturbations](https://mlsanity.com/paper/2103.14641/) (2021)
- [Analysis and Applications of Class-wise Robustness in Adversarial Training](https://mlsanity.com/paper/2105.14240/) (2021)
- [A Geometric Perspective on the Transferability of Adversarial Directions](https://mlsanity.com/paper/1811.03531/) (2018)

## Built on by (most cited first, 22 of 22)

- [A Survey on Transferability of Adversarial Examples across Deep Neural Networks](https://mlsanity.com/paper/2310.17626/) (2023)
- [SA-Attack: Improving Adversarial Transferability of Vision-Language Pre-training Models via Self-Augmentation](https://mlsanity.com/paper/2312.04913/) (2023)
- [OT-Attack: Enhancing Adversarial Transferability of Vision-Language Models via Optimal Transport Optimization](https://mlsanity.com/paper/2312.04403/) (2023)
- [Reliable Evaluation of Adversarial Transferability](https://mlsanity.com/paper/2306.08565/) (2023)
- [As Firm As Their Foundations: Can open-sourced foundation models be used to create adversarial examples for downstream tasks?](https://mlsanity.com/paper/2403.12693/) (2024)
- [Improving Adversarial Transferability via Model Alignment](https://mlsanity.com/paper/2311.18495/) (2023)
- [Similarity of Neural Architectures using Adversarial Attack Transferability](https://mlsanity.com/paper/2210.11407/) (2022)
- [On the Robustness of Tabular Foundation Models: Test-Time Attacks and In-Context Defenses](https://mlsanity.com/paper/2506.02978/) (2025)
- [Evasion Adversarial Attacks Remain Impractical Against ML-based Network Intrusion Detection Systems, Especially Dynamic Ones](https://mlsanity.com/paper/2306.05494/) (2023)
- [Clean Label Attacks against SLU Systems](https://mlsanity.com/paper/2409.08985/) (2024)
- [Merge Now, Regret Later: The Hidden Cost of Model Merging Is Adversarial Transferability](https://mlsanity.com/paper/2509.23689/) (2025)
- [Adversarial-Aware Deep Learning System based on a Secondary Classical Machine Learning Verification Approach](https://mlsanity.com/paper/2306.00314/) (2023)
- [Common Knowledge Learning for Generating Transferable Adversarial Examples](https://mlsanity.com/paper/2307.00274/) (2023)
- [Towards Understanding Adversarial Transferability in Federated Learning](https://mlsanity.com/paper/2310.00616/) (2023)
- [Understanding Adversarial Transfer: Why Representation-Space Attacks Fail Where Data-Space Attacks Succeed](https://mlsanity.com/paper/2510.01494/) (2025)
- [On the Computational Entanglement of Distant Features in Adversarial Machine Learning](https://mlsanity.com/paper/2309.15669/) (2023)
- [A Framework for Adversarial Analysis of Decision Support Systems Prior to Deployment](https://mlsanity.com/paper/2505.21414/) (2025)
- [Exploring Cross-Stage Adversarial Transferability in Class-Incremental Continual Learning](https://mlsanity.com/paper/2508.08920/) (2025)
- [IGAff: Benchmarking Adversarial Iterative and Genetic Affine Algorithms on Deep Neural Networks](https://mlsanity.com/paper/2509.06459/) (2025)
- [Adversarial Vulnerability Transcends Computational Paradigms: Feature Engineering Provides No Defense Against Neural Adversarial Transfer](https://mlsanity.com/paper/2601.21323/) (2026)
- [Break the Brake, Not the Wheel: Untargeted Jailbreak via Entropy Maximization](https://mlsanity.com/paper/2605.10764/) (2026)
- [Exploring Solver-Level Warmstarting for Neural Network Verification](https://mlsanity.com/paper/2609.25962/) (2026)

## Abstract (arXiv)

Deep neural networks are vulnerable to adversarial examples (AEs), which have adversarial transferability: AEs generated for the source model can mislead another (target) model's predictions. However, the transferability has not been understood in terms of to which class target model's predictions were misled (i.e., class-aware transferability). In this paper, we differentiate the cases in which a target model predicts the same wrong class as the source model ("same mistake") or a different wrong class ("different mistake") to analyze and provide an explanation of the mechanism. We find that (1) AEs tend to cause same mistakes, which correlates with "non-targeted transferability"; however, (2) different mistakes occur even between similar models, regardless of the perturbation size. Furthermore, we present evidence that the difference between same mistakes and different mistakes can be explained by non-robust features, predictive but human-uninterpretable patterns: different mistakes occur when non-robust features in AEs are used differently by models. Non-robust features can thus provide consistent explanations for the class-aware transferability of AEs.