# Saliency Attack: Towards Imperceptible Black-box Adversarial Attack

arXiv: https://arxiv.org/abs/2206.01898 · HTML: https://mlsanity.com/paper/2206.01898/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 51 references that are in the map.
- First posted on arXiv 4 Jun 2022.

## Builds on (most cited first, 30 of 33)

- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Towards Evaluating the Robustness of Neural Networks](https://mlsanity.com/paper/1608.04644/) (2016)
- [Learning Deep Features for Discriminative Localization](https://mlsanity.com/paper/1512.04150/) (2015)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [DeepFool: a simple and accurate method to fool deep neural networks](https://mlsanity.com/paper/1511.04599/) (2015)
- [The Limitations of Deep Learning in Adversarial Settings](https://mlsanity.com/paper/1511.07528/) (2015)
- [Square Attack: a query-efficient black-box adversarial attack via random search](https://mlsanity.com/paper/1912.00049/) (2019)
- [ZOO: Zeroth Order Optimization based Black-box Attacks to Deep Neural Networks without Training Substitute Models](https://mlsanity.com/paper/1708.03999/) (2017)
- [One pixel attack for fooling deep neural networks](https://mlsanity.com/paper/1710.08864/) (2017)
- [Black-box Adversarial Attacks with Limited Queries and Information](https://mlsanity.com/paper/1804.08598/) (2018)
- [Feature Squeezing: Detecting Adversarial Examples in Deep Neural Networks](https://mlsanity.com/paper/1704.01155/) (2017)
- [Decision-Based Adversarial Attacks: Reliable Attacks Against Black-Box Machine Learning Models](https://mlsanity.com/paper/1712.04248/) (2017)
- [Prior Convictions: Black-Box Adversarial Attacks with Bandits and Priors](https://mlsanity.com/paper/1807.07978/) (2018)
- [AutoZOOM: Autoencoder-based Zeroth Order Optimization Method for Attacking Black-box Neural Networks](https://mlsanity.com/paper/1805.11770/) (2018)
- [Pyramid Feature Attention Network for Saliency detection](https://mlsanity.com/paper/1903.00179/) (2019)
- [Sparse and Imperceivable Adversarial Attacks](https://mlsanity.com/paper/1909.05040/) (2019)
- [GenAttack: Practical Black-box Attacks with Gradient-Free Optimization](https://mlsanity.com/paper/1805.11090/) (2018)
- [Low Frequency Adversarial Perturbation](https://mlsanity.com/paper/1809.08758/) (2018)
- [Towards Large yet Imperceptible Adversarial Image Perturbations with Perceptual Color Distance](https://mlsanity.com/paper/1911.02466/) (2019)
- [On the Suitability of Lp-norms for Creating and Preventing Adversarial Examples](https://mlsanity.com/paper/1802.09653/) (2018)
- [Parsimonious Black-Box Adversarial Attacks via Efficient Combinatorial Optimization](https://mlsanity.com/paper/1905.06635/) (2019)
- [Demiguise Attack: Crafting Invisible Semantic Adversarial Perturbations with Perceptual Similarity](https://mlsanity.com/paper/2107.01396/) (2021)
- [Efficient Combinatorial Optimization for Word-level Adversarial Textual Attack](https://mlsanity.com/paper/2109.02229/) (2021)
- [Perceptual Evaluation of Adversarial Attacks for CNN-based Image Classification](https://mlsanity.com/paper/1906.00204/) (2019)

## Built on by (most cited first, 12 of 12)

- [Imperceptible Protection against Style Imitation from Diffusion Models](https://mlsanity.com/paper/2403.19254/) (2024)
- [An adversarial attack approach for eXplainable AI evaluation on deepfake detection models](https://mlsanity.com/paper/2312.06627/) (2023)
- [Nearly Zero-Cost Protection Against Mimicry by Personalized Diffusion Models](https://mlsanity.com/paper/2412.11423/) (2024)
- [Reliable Robustness Evaluation via Automatically Constructed Attack Ensembles](https://mlsanity.com/paper/2211.12713/) (2022)
- [StealthAttack: Robust 3D Gaussian Splatting Poisoning via Density-Guided Illusions](https://mlsanity.com/paper/2510.02314/) (2025)
- [SUNY: A Visual Interpretation Framework for Convolutional Neural Networks from a Necessary and Sufficient Perspective](https://mlsanity.com/paper/2303.00244/) (2023)
- [SemDiff: Generating Natural Unrestricted Adversarial Examples via Semantic Attributes Optimization in Diffusion Models](https://mlsanity.com/paper/2504.11923/) (2025)
- [Perturbation-Based Two-Stage Multi-Domain Active Learning](https://mlsanity.com/paper/2306.10700/) (2023)
- [Bridging the Gap Between Saliency Prediction and Image Quality Assessment](https://mlsanity.com/paper/2405.04997/) (2024)
- [Out-of-the-box: Black-box Causal Attacks on Object Detectors](https://mlsanity.com/paper/2512.03730/) (2025)
- [Superpixel Attack: Enhancing Black-box Adversarial Attack with Image-driven Division Areas](https://mlsanity.com/paper/2512.02062/) (2025)
- [Don't Waste the Noise: Importance-Guided Perturbation Allocation under Joint Global and Local Constraints](https://mlsanity.com/paper/2610.00861/) (2026)

## Abstract (arXiv)

Deep neural networks are vulnerable to adversarial examples, even in the black-box setting where the attacker is only accessible to the model output. Recent studies have devised effective black-box attacks with high query efficiency. However, such performance is often accompanied by compromises in attack imperceptibility, hindering the practical use of these approaches. In this paper, we propose to restrict the perturbations to a small salient region to generate adversarial examples that can hardly be perceived. This approach is readily compatible with many existing black-box attacks and can significantly improve their imperceptibility with little degradation in attack success rate. Further, we propose the Saliency Attack, a new black-box attack aiming to refine the perturbations in the salient region to achieve even better imperceptibility. Extensive experiments show that compared to the state-of-the-art black-box attacks, our approach achieves much better imperceptibility scores, including most apparent distortion (MAD), L0 and L2 distances, and also obtains significantly higher success rates judged by a human-like threshold on MAD. Importantly, the perturbations generated by our approach are interpretable to some extent. Finally, it is also demonstrated to be robust to different detection-based defenses.