# Simple Black-Box Adversarial Perturbations for Deep Networks

arXiv: https://arxiv.org/abs/1612.06299 · HTML: https://mlsanity.com/paper/1612.06299/

## Key facts

- Cited by 42 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Dec 2016.

## Built on by (most cited first, 30 of 42)

- [Diffusion Models for Imperceptible and Transferable Adversarial Attack](https://mlsanity.com/paper/2305.08192/) (2023)
- [Black Box Adversarial Prompting for Foundation Models](https://mlsanity.com/paper/2302.04237/) (2023)
- [Improving the Transferability of Adversarial Examples with Resized-Diverse-Inputs, Diversity-Ensemble and Region Fitting](https://mlsanity.com/paper/2112.06011/) (2021)
- [Inspect, Understand, Overcome: A Survey of Practical Methods for AI Safety](https://mlsanity.com/paper/2104.14235/) (2021)
- [Towards Trustworthy and Aligned Machine Learning: A Data-centric Survey with Causality Perspectives](https://mlsanity.com/paper/2307.16851/) (2023)
- [Understanding the Dynamics of DNNs Using Graph Modularity](https://mlsanity.com/paper/2111.12485/) (2021)
- [Adversarial Attacks of Vision Tasks in the Past 10 Years: A Survey](https://mlsanity.com/paper/2410.23687/) (2024)
- [Attacks in Adversarial Machine Learning: A Systematic Survey from the Life-cycle Perspective](https://mlsanity.com/paper/2302.09457/) (2023)
- [Physical Adversarial Attacks For Camera-based Smart Systems: Current Trends, Categorization, Applications, Research Challenges, and Future Outlook](https://mlsanity.com/paper/2308.06173/) (2023)
- [Towards Adversarially Robust Dataset Distillation by Curvature Regularization](https://mlsanity.com/paper/2403.10045/) (2024)
- [Adversarial Imaging Pipelines](https://mlsanity.com/paper/2102.03728/) (2021)
- [AdvART: Adversarial Art for Camouflaged Object Detection Attacks](https://mlsanity.com/paper/2303.01734/) (2023)
- [SAAM: Stealthy Adversarial Attack on Monocular Depth Estimation](https://mlsanity.com/paper/2308.03108/) (2023)
- [Black-box adversarial attacks using Evolution Strategies](https://mlsanity.com/paper/2104.15064/) (2021)
- [Federated Robustness Propagation: Sharing Robustness in Heterogeneous Federated Learning](https://mlsanity.com/paper/2106.10196/) (2021)
- [ADI: Adversarial Dominating Inputs in Vertical Federated Learning Systems](https://mlsanity.com/paper/2201.02775/) (2022)
- [AdvRain: Adversarial Raindrops to Attack Camera-based Smart Vision Systems](https://mlsanity.com/paper/2303.01338/) (2023)
- [DST: Dynamic Substitute Training for Data-free Black-box Attack](https://mlsanity.com/paper/2204.00972/) (2022)
- [Assessing Robustness via Score-Based Adversarial Image Generation](https://mlsanity.com/paper/2310.04285/) (2023)
- [GSE: Group-wise Sparse and Explainable Adversarial Attacks](https://mlsanity.com/paper/2311.17434/) (2023)
- [Adversarial Attacks and Defenses on 3D Point Cloud Classification: A Survey](https://mlsanity.com/paper/2307.00309/) (2023)
- [Exploiting Vulnerabilities in Deep Neural Networks: Adversarial and Fault-Injection Attacks](https://mlsanity.com/paper/2105.03251/) (2021)
- [Mitigating Black-Box Adversarial Attacks via Output Noise Perturbation](https://mlsanity.com/paper/2109.15160/) (2021)
- [Query-Efficient Adversarial Attack Based on Latin Hypercube Sampling](https://mlsanity.com/paper/2207.02391/) (2022)
- [Towards Non-Adversarial Algorithmic Recourse](https://mlsanity.com/paper/2403.10330/) (2024)
- [A Red Teaming Roadmap Towards System-Level Safety](https://mlsanity.com/paper/2506.05376/) (2025)
- [Adversarial training may be a double-edged sword](https://mlsanity.com/paper/2107.11671/) (2021)
- [Efficient and Robust Classification for Sparse Attacks](https://mlsanity.com/paper/2201.09369/) (2022)
- [Salient Feature Extractor for Adversarial Defense on Deep Neural Networks](https://mlsanity.com/paper/2105.06807/) (2021)
- [Output Randomization: A Novel Defense for both White-box and Black-box Adversarial Models](https://mlsanity.com/paper/2107.03806/) (2021)

## Abstract (arXiv)

Deep neural networks are powerful and popular learning models that achieve state-of-the-art pattern recognition performance on many computer vision, speech, and language processing tasks. However, these networks have also been shown susceptible to carefully crafted adversarial perturbations which force misclassification of the inputs. Adversarial examples enable adversaries to subvert the expected system behavior leading to undesired consequences and could pose a security risk when these systems are deployed in the real world. In this work, we focus on deep convolutional neural networks and demonstrate that adversaries can easily craft adversarial examples even without any internal knowledge of the target network. Our attacks treat the network as an oracle (black-box) and only assume that the output of the network can be observed on the probed inputs. Our first attack is based on a simple idea of adding perturbation to a randomly selected single pixel or a small set of them. We then improve the effectiveness of this attack by carefully constructing a small set of pixels to perturb by using the idea of greedy local-search. Our proposed attacks also naturally extend to a stronger notion of misclassification. Our extensive experimental results illustrate that even these elementary attacks can reveal a deep neural network's vulnerabilities. The simplicity and effectiveness of our proposed schemes mean that they could serve as a litmus test for designing robust networks.