# Disrupting Adversarial Transferability in Deep Neural Networks

arXiv: https://arxiv.org/abs/2108.12492 · HTML: https://mlsanity.com/paper/2108.12492/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 31 references that are in the map.
- First posted on arXiv 27 Aug 2021.

## Builds on (most cited first, 23 of 23)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Towards Evaluating the Robustness of Neural Networks](https://mlsanity.com/paper/1608.04644/) (2016)
- [Adversarial examples in the physical world](https://mlsanity.com/paper/1607.02533/) (2016)
- [Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples](https://mlsanity.com/paper/1802.00420/) (2018)
- [Adversarial Examples Are Not Bugs, They Are Features](https://mlsanity.com/paper/1905.02175/) (2019)
- [The Limitations of Deep Learning in Adversarial Settings](https://mlsanity.com/paper/1511.07528/) (2015)
- [Universal adversarial perturbations](https://mlsanity.com/paper/1610.08401/) (2016)
- [Distillation as a Defense to Adversarial Perturbations against Deep Neural Networks](https://mlsanity.com/paper/1511.04508/) (2015)
- [Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples](https://mlsanity.com/paper/1605.07277/) (2016)
- [Audio Adversarial Examples: Targeted Attacks on Speech-to-Text](https://mlsanity.com/paper/1801.01944/) (2018)
- [On instabilities of deep learning in image reconstruction - Does AI come at a cost?](https://mlsanity.com/paper/1902.05300/) (2019)
- [Vision Transformers are Robust Learners](https://mlsanity.com/paper/2105.07581/) (2021)
- [The Space of Transferable Adversarial Examples](https://mlsanity.com/paper/1704.03453/) (2017)
- [Constructing Unrestricted Adversarial Examples with Generative Models](https://mlsanity.com/paper/1805.07894/) (2018)
- [DVERGE: Diversifying Vulnerabilities for Enhanced Robust Generation of Ensembles](https://mlsanity.com/paper/2009.14720/) (2020)
- [Defensive Distillation is Not Robust to Adversarial Examples](https://mlsanity.com/paper/1607.04311/) (2016)
- [Universal Adversarial Perturbations: A Survey](https://mlsanity.com/paper/2005.08087/) (2020)
- [Generative Adversarial Trainer: Defense to Adversarial Perturbations with GAN](https://mlsanity.com/paper/1705.03387/) (2017)
- [High Dimensional Spaces, Deep Learning and Adversarial Examples](https://mlsanity.com/paper/1801.00634/) (2018)

## Built on by (most cited first, 6 of 6)

- [Adversarial Attacks Leverage Interference Between Features in Superposition](https://mlsanity.com/paper/2510.11709/) (2025)
- [Evaluating Similitude and Robustness of Deep Image Denoising Models via Adversarial Attack](https://mlsanity.com/paper/2306.16050/) (2023)
- [Towards Scalable and Robust Model Versioning](https://mlsanity.com/paper/2401.09574/) (2024)
- [Understanding Adversarial Transfer: Why Representation-Space Attacks Fail Where Data-Space Attacks Succeed](https://mlsanity.com/paper/2510.01494/) (2025)
- [Decorrelative Network Architecture for Robust Electrocardiogram Classification](https://mlsanity.com/paper/2207.09031/) (2022)
- [Robust Subgraph Learning by Monitoring Early Training Representations](https://mlsanity.com/paper/2403.09901/) (2024)

## Abstract (arXiv)

Adversarial attack transferability is well-recognized in deep learning. Prior work has partially explained transferability by recognizing common adversarial subspaces and correlations between decision boundaries, but little is known beyond this. We propose that transferability between seemingly different models is due to a high linear correlation between the feature sets that different networks extract. In other words, two models trained on the same task that are distant in the parameter space likely extract features in the same fashion, just with trivial affine transformations between the latent spaces. Furthermore, we show how applying a feature correlation loss, which decorrelates the extracted features in a latent space, can reduce the transferability of adversarial attacks between models, suggesting that the models complete tasks in semantically different ways. Finally, we propose a Dual Neck Autoencoder (DNA), which leverages this feature correlation loss to create two meaningfully different encodings of input information with reduced transferability.