# On the Effectiveness of Small Input Noise for Defending Against Query-based Black-Box Attacks

arXiv: https://arxiv.org/abs/2101.04829 · HTML: https://mlsanity.com/paper/2101.04829/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 43 references that are in the map.
- First posted on arXiv 13 Jan 2021.

## Builds on (most cited first, 30 of 32)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Identity Mappings in Deep Residual Networks](https://mlsanity.com/paper/1603.05027/) (2016)
- [Certified Adversarial Robustness via Randomized Smoothing](https://mlsanity.com/paper/1902.02918/) (2019)
- [Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples](https://mlsanity.com/paper/1802.00420/) (2018)
- [Practical Black-Box Attacks against Machine Learning](https://mlsanity.com/paper/1602.02697/) (2016)
- [Robustness May Be at Odds with Accuracy](https://mlsanity.com/paper/1805.12152/) (2018)
- [Ensemble Adversarial Training: Attacks and Defenses](https://mlsanity.com/paper/1705.07204/) (2017)
- [Evasion Attacks against Machine Learning at Test Time](https://mlsanity.com/paper/1708.06131/) (2017)
- [ZOO: Zeroth Order Optimization based Black-box Attacks to Deep Neural Networks without Training Substitute Models](https://mlsanity.com/paper/1708.03999/) (2017)
- [Decision-Based Adversarial Attacks: Reliable Attacks Against Black-Box Machine Learning Models](https://mlsanity.com/paper/1712.04248/) (2017)
- [On Evaluating Adversarial Robustness](https://mlsanity.com/paper/1902.06705/) (2019)
- [Mitigating Adversarial Effects Through Randomization](https://mlsanity.com/paper/1711.01991/) (2017)
- [HopSkipJumpAttack: A Query-Efficient Decision-Based Attack](https://mlsanity.com/paper/1904.02144/) (2019)
- [Simple Black-box Adversarial Attacks](https://mlsanity.com/paper/1905.07121/) (2019)
- [Adversarial Robustness Toolbox v1.0.0](https://mlsanity.com/paper/1807.01069/) (2018)
- [Certified Adversarial Robustness with Additive Noise](https://mlsanity.com/paper/1809.03113/) (2018)
- [Towards Robust Neural Networks via Random Self-ensemble](https://mlsanity.com/paper/1712.00673/) (2017)
- [Towards Deep Neural Network Architectures Robust to Adversarial Examples](https://mlsanity.com/paper/1412.5068/) (2014)
- [Prior Convictions: Black-Box Adversarial Attacks with Bandits and Priors](https://mlsanity.com/paper/1807.07978/) (2018)
- [Query-Efficient Hard-label Black-box Attack:An Optimization-based Approach](https://mlsanity.com/paper/1807.04457/) (2018)
- [Parametric Noise Injection: Trainable Randomness to Improve Deep Neural Network Robustness against Adversarial Attack](https://mlsanity.com/paper/1811.09310/) (2018)
- [Sign-OPT: A Query-Efficient Hard-label Adversarial Attack](https://mlsanity.com/paper/1909.10773/) (2019)
- [Low Frequency Adversarial Perturbation](https://mlsanity.com/paper/1809.08758/) (2018)
- [GeoDA: a geometric framework for black-box adversarial attacks](https://mlsanity.com/paper/2003.06468/) (2020)
- [Stateful Detection of Black-Box Adversarial Attacks](https://mlsanity.com/paper/1907.05587/) (2019)
- [A Survey of Black-Box Adversarial Attacks on Computer Vision Models](https://mlsanity.com/paper/1912.01667/) (2019)
- [Subspace Attack: Exploiting Promising Subspaces for Query-Efficient Black-box Attacks](https://mlsanity.com/paper/1906.04392/) (2019)
- [Theoretical evidence for adversarial robustness through randomization](https://mlsanity.com/paper/1902.01148/) (2019)

## Built on by (most cited first, 17 of 17)

- [Deepfake Text Detection: Limitations and Opportunities](https://mlsanity.com/paper/2210.09421/) (2022)
- [Defenses in Adversarial Machine Learning: A Survey](https://mlsanity.com/paper/2312.08890/) (2023)
- [Improving the Robustness of Transformer-based Large Language Models with Dynamic Attention](https://mlsanity.com/paper/2311.17400/) (2023)
- [Improving LLM Unlearning Robustness via Random Perturbations](https://mlsanity.com/paper/2501.19202/) (2025)
- [Robust Prototypical Few-Shot Organ Segmentation with Regularized Neural-ODEs](https://mlsanity.com/paper/2208.12428/) (2022)
- [The Adaptive Arms Race: Redefining Robustness in AI Security](https://mlsanity.com/paper/2312.13435/) (2023)
- [Understanding the Robustness of Randomized Feature Defense Against Query-Based Adversarial Attacks](https://mlsanity.com/paper/2310.00567/) (2023)
- [Merge Now, Regret Later: The Hidden Cost of Model Merging Is Adversarial Transferability](https://mlsanity.com/paper/2509.23689/) (2025)
- [Unifying Gradients to Improve Real-world Robustness for Deep Networks](https://mlsanity.com/paper/2208.06228/) (2022)
- [Guidance Through Surrogate: Towards a Generic Diagnostic Attack](https://mlsanity.com/paper/2212.14875/) (2022)
- [Exploring Non-additive Randomness on ViT against Query-Based Black-Box Attacks](https://mlsanity.com/paper/2309.06438/) (2023)
- [A Training Rate and Survival Heuristic for Inference and Robustness Evaluation (TRASHFIRE)](https://mlsanity.com/paper/2401.13751/) (2024)
- [How stealthy is stealthy? Studying the Efficacy of Black-Box Adversarial Attacks in the Real World](https://mlsanity.com/paper/2506.05382/) (2025)
- [SEA: Shareable and Explainable Attribution for Query-based Black-box Attacks](https://mlsanity.com/paper/2308.11845/) (2023)
- [Counter-Samples: A Stateless Strategy to Neutralize Black Box Adversarial Attacks](https://mlsanity.com/paper/2403.10562/) (2024)
- [Addressing Key Challenges of Adversarial Attacks and Defenses in the Tabular Domain: A Methodological Framework for Coherence and Consistency](https://mlsanity.com/paper/2412.07326/) (2024)
- [Dashed Line Defense: Plug-And-Play Defense Against Adaptive Score-Based Query Attacks](https://mlsanity.com/paper/2602.08679/) (2026)

## Abstract (arXiv)

While deep neural networks show unprecedented performance in various tasks, the vulnerability to adversarial examples hinders their deployment in safety-critical systems. Many studies have shown that attacks are also possible even in a black-box setting where an adversary cannot access the target model's internal information. Most black-box attacks are based on queries, each of which obtains the target model's output for an input, and many recent studies focus on reducing the number of required queries. In this paper, we pay attention to an implicit assumption of query-based black-box adversarial attacks that the target model's output exactly corresponds to the query input. If some randomness is introduced into the model, it can break the assumption, and thus, query-based attacks may have tremendous difficulty in both gradient estimation and local search, which are the core of their attack process. From this motivation, we observe even a small additive input noise can neutralize most query-based attacks and name this simple yet effective approach Small Noise Defense (SND). We analyze how SND can defend against query-based black-box attacks and demonstrate its effectiveness against eight state-of-the-art attacks with CIFAR-10 and ImageNet datasets. Even with strong defense ability, SND almost maintains the original classification accuracy and computational speed. SND is readily applicable to pre-trained models by adding only one line of code at the inference.