# On Generating Transferable Targeted Perturbations

arXiv: https://arxiv.org/abs/2103.14641 · HTML: https://mlsanity.com/paper/2103.14641/

## Key facts

- Cited by 65 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 48 references that are in the map.
- First posted on arXiv 26 Mar 2021.

## Builds on (most cited first, 30 of 35)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness](https://mlsanity.com/paper/1811.12231/) (2018)
- [Transformers in Vision: A Survey](https://mlsanity.com/paper/2101.01169/) (2021)
- [Adversarial Examples Are Not Bugs, They Are Features](https://mlsanity.com/paper/1905.02175/) (2019)
- [AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty](https://mlsanity.com/paper/1912.02781/) (2019)
- [Universal adversarial perturbations](https://mlsanity.com/paper/1610.08401/) (2016)
- [Improving Transferability of Adversarial Examples with Input Diversity](https://mlsanity.com/paper/1803.06978/) (2018)
- [Delving into Transferable Adversarial Examples and Black-box Attacks](https://mlsanity.com/paper/1611.02770/) (2016)
- [Evading Defenses to Transferable Adversarial Examples by Translation-Invariant Attacks](https://mlsanity.com/paper/1904.02884/) (2019)
- [Do Adversarially Robust ImageNet Models Transfer Better?](https://mlsanity.com/paper/2007.08489/) (2020)
- [A Self-supervised Approach for Adversarial Robustness](https://mlsanity.com/paper/2006.04924/) (2020)
- [Skip Connections Matter: On the Transferability of Adversarial Examples Generated with ResNets](https://mlsanity.com/paper/2002.05990/) (2020)
- [Generative Adversarial Perturbations](https://mlsanity.com/paper/1712.02328/) (2017)
- [Enhancing Adversarial Example Transferability with an Intermediate Level Attack](https://mlsanity.com/paper/1907.10823/) (2019)
- [Adversarial Examples Are a Natural Consequence of Test Error in Noise](https://mlsanity.com/paper/1901.10513/) (2019)
- [Feature Distillation: DNN-Oriented JPEG Compression Against Adversarial Examples](https://mlsanity.com/paper/1803.05787/) (2018)
- [Cross-Domain Transferability of Adversarial Perturbations](https://mlsanity.com/paper/1905.11736/) (2019)
- [Generalizable Data-free Objective for Crafting Universal Adversarial Perturbations](https://mlsanity.com/paper/1801.08092/) (2018)
- [Local Gradients Smoothing: Defense against localized adversarial attacks](https://mlsanity.com/paper/1807.01216/) (2018)
- [Backpropagating Linearly Improves Transferability of Adversarial Examples](https://mlsanity.com/paper/2012.03528/) (2020)
- [Shield: Fast, Practical Defense and Vaccination for Deep Learning using JPEG Compression](https://mlsanity.com/paper/1802.06816/) (2018)
- [Fast Feature Fool: A data independent approach to universal adversarial perturbations](https://mlsanity.com/paper/1707.05572/) (2017)
- [Perturbing Across the Feature Hierarchy to Improve Standard and Strict Blackbox Attack Transferability](https://mlsanity.com/paper/2004.14861/) (2020)

## Built on by (most cited first, 30 of 65)

- [Rethinking Model Ensemble in Transfer-based Adversarial Attacks](https://mlsanity.com/paper/2303.09105/) (2023)
- [On Improving Adversarial Transferability of Vision Transformers](https://mlsanity.com/paper/2106.04169/) (2021)
- [Boosting the Transferability of Adversarial Attacks with Reverse Adversarial Perturbation](https://mlsanity.com/paper/2210.05968/) (2022)
- [A Survey on Transferability of Adversarial Examples across Deep Neural Networks](https://mlsanity.com/paper/2310.17626/) (2023)
- [Towards Understanding and Boosting Adversarial Transferability from a Distribution Perspective](https://mlsanity.com/paper/2210.04213/) (2022)
- [Boosting Transferability of Targeted Adversarial Examples via Hierarchical Generative Networks](https://mlsanity.com/paper/2107.01809/) (2021)
- [Why Does Little Robustness Help? A Further Step Towards Understanding Adversarial Transferability](https://mlsanity.com/paper/2307.07873/) (2023)
- [Enhancing the Self-Universality for Transferable Targeted Attacks](https://mlsanity.com/paper/2209.03716/) (2022)
- [Towards Transferable Targeted 3D Adversarial Attack in the Physical World](https://mlsanity.com/paper/2312.09558/) (2023)
- [Closer Look at the Transferability of Adversarial Examples: How They Fool Different Models Differently](https://mlsanity.com/paper/2112.14337/) (2021)
- [Attacks in Adversarial Machine Learning: A Systematic Survey from the Life-cycle Perspective](https://mlsanity.com/paper/2302.09457/) (2023)
- [CLIP-Guided Generative Networks for Transferable Targeted Adversarial Attacks](https://mlsanity.com/paper/2407.10179/) (2024)
- [Logit Margin Matters: Improving Transferable Targeted Adversarial Attack by Logit Calibration](https://mlsanity.com/paper/2303.03680/) (2023)
- [Exploring Transferability of Multimodal Adversarial Samples for Vision-Language Pre-training Models with Contrastive Learning](https://mlsanity.com/paper/2308.12636/) (2023)
- [Towards Good Practices in Evaluating Transfer Adversarial Attacks](https://mlsanity.com/paper/2211.09565/) (2022)
- [One Surrogate to Fool Them All: Universal, Transferable, and Targeted Adversarial Attacks with CLIP](https://mlsanity.com/paper/2505.19840/) (2025)
- [Minimizing Maximum Model Discrepancy for Transferable Black-box Targeted Attacks](https://mlsanity.com/paper/2212.09035/) (2022)
- [Staircase Sign Method for Boosting Adversarial Attacks](https://mlsanity.com/paper/2104.09722/) (2021)
- [AdvI2I: Adversarial Image Attack on Image-to-Image Diffusion models](https://mlsanity.com/paper/2410.21471/) (2024)
- [Prompt-Driven Contrastive Learning for Transferable Adversarial Attacks](https://mlsanity.com/paper/2407.20657/) (2024)
- [FACL-Attack: Frequency-Aware Contrastive Learning for Transferable Adversarial Attacks](https://mlsanity.com/paper/2407.20653/) (2024)
- [Enhancing targeted transferability via feature space fine-tuning](https://mlsanity.com/paper/2401.02727/) (2024)
- [Adversarial Pixel Restoration as a Pretext Task for Transferable Perturbations](https://mlsanity.com/paper/2207.08803/) (2022)
- [SoK: Pitfalls in Evaluating Black-Box Attacks](https://mlsanity.com/paper/2310.17534/) (2023)
- [Towards Evaluating Transfer-based Attacks Systematically, Practically, and Fairly](https://mlsanity.com/paper/2311.01323/) (2023)
- [Simulations of Common Unsupervised Domain Adaptation Algorithms for Image Classification](https://mlsanity.com/paper/2502.10694/) (2025)
- [Scale-free and Task-agnostic Attack: Generating Photo-realistic Adversarial Patterns with Patch Quilting Generator](https://mlsanity.com/paper/2208.06222/) (2022)
- [Boosting Adversarial Transferability using Dynamic Cues](https://mlsanity.com/paper/2302.12252/) (2023)
- [A Comprehensive Study on the Robustness of Image Classification and Object Detection in Remote Sensing: Surveying and Benchmarking](https://mlsanity.com/paper/2306.12111/) (2023)
- [Low-Mid Adversarial Perturbation against Unauthorized Face Recognition System](https://mlsanity.com/paper/2206.09410/) (2022)

## Abstract (arXiv)

While the untargeted black-box transferability of adversarial perturbations has been extensively studied before, changing an unseen model's decisions to a specific `targeted' class remains a challenging feat. In this paper, we propose a new generative approach for highly transferable targeted perturbations (ours). We note that the existing methods are less suitable for this task due to their reliance on class-boundary information that changes from one model to another, thus reducing transferability. In contrast, our approach matches the perturbed image `distribution' with that of the target class, leading to high targeted transferability rates. To this end, we propose a new objective function that not only aligns the global distributions of source and target images, but also matches the local neighbourhood structure between the two domains. Based on the proposed objective, we train a generator function that can adaptively synthesize perturbations specific to a given input. Our generative approach is independent of the source or target domain labels, while consistently performs well against state-of-the-art methods on a wide range of attack settings. As an example, we achieve 32.63% target transferability from (an adversarially weak) VGG19BN to (a strong) WideResNet on ImageNet val. set, which is 4× higher than the previous best generative attack and 16× better than instance-specific iterative attack. Code is available at: smallurlhttps://github.com/Muzammal-Naseer/TTP.