# Detecting Adversarial Examples Is (Nearly) As Hard As Classifying Them

arXiv: https://arxiv.org/abs/2107.11630 · HTML: https://mlsanity.com/paper/2107.11630/

## Key facts

- Cited by 48 papers in the map (data through 14 Oct 2026).
- Builds on 40 of its 48 references that are in the map.
- First posted on arXiv 24 Jul 2021.

## Builds on (most cited first, 30 of 40)

- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Theoretically Principled Trade-off between Robustness and Accuracy](https://mlsanity.com/paper/1901.08573/) (2019)
- [A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks](https://mlsanity.com/paper/1807.03888/) (2018)
- [Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks](https://mlsanity.com/paper/2003.01690/) (2020)
- [Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples](https://mlsanity.com/paper/1802.00420/) (2018)
- [Robustness May Be at Odds with Accuracy](https://mlsanity.com/paper/1805.12152/) (2018)
- [Evasion Attacks against Machine Learning at Test Time](https://mlsanity.com/paper/1708.06131/) (2017)
- [Square Attack: a query-efficient black-box adversarial attack via random search](https://mlsanity.com/paper/1912.00049/) (2019)
- [RobustBench: a standardized adversarial robustness benchmark](https://mlsanity.com/paper/2010.09670/) (2020)
- [Provable defenses against adversarial examples via the convex outer adversarial polytope](https://mlsanity.com/paper/1711.00851/) (2017)
- [On Adaptive Attacks to Adversarial Example Defenses](https://mlsanity.com/paper/2002.08347/) (2020)
- [Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methods](https://mlsanity.com/paper/1705.07263/) (2017)
- [Adversarial Patch](https://mlsanity.com/paper/1712.09665/) (2017)
- [Feature Squeezing: Detecting Adversarial Examples in Deep Neural Networks](https://mlsanity.com/paper/1704.01155/) (2017)
- [On Evaluating Adversarial Robustness](https://mlsanity.com/paper/1902.06705/) (2019)
- [Unlabeled Data Improves Adversarial Robustness](https://mlsanity.com/paper/1905.13736/) (2019)
- [Adversarially Robust Generalization Requires More Data](https://mlsanity.com/paper/1804.11285/) (2018)
- [Certified Defenses against Adversarial Examples](https://mlsanity.com/paper/1801.09344/) (2018)
- [Characterizing Adversarial Subspaces Using Local Intrinsic Dimensionality](https://mlsanity.com/paper/1801.02613/) (2018)
- [Detecting Adversarial Samples from Artifacts](https://mlsanity.com/paper/1703.00410/) (2017)
- [Towards Stable and Efficient Training of Verifiably Robust Neural Networks](https://mlsanity.com/paper/1906.06316/) (2019)
- [Fixing Data Augmentation to Improve Adversarial Robustness](https://mlsanity.com/paper/2103.01946/) (2021)
- [Adversarial Training and Robustness for Multiple Perturbations](https://mlsanity.com/paper/1904.13000/) (2019)
- [On the (Statistical) Detection of Adversarial Examples](https://mlsanity.com/paper/1702.06280/) (2017)
- [Are Labels Required for Improving Adversarial Robustness?](https://mlsanity.com/paper/1905.13725/) (2019)
- [Certified Defenses for Adversarial Patches](https://mlsanity.com/paper/2003.06693/) (2020)
- [Local Gradients Smoothing: Defense against localized adversarial attacks](https://mlsanity.com/paper/1807.01216/) (2018)
- [Adversarially Robust Generalization Just Requires More Unlabeled Data](https://mlsanity.com/paper/1906.00555/) (2019)
- [The Odds are Odd: A Statistical Test for Detecting Adversarial Examples](https://mlsanity.com/paper/1902.04818/) (2019)

## Built on by (most cited first, 30 of 48)

- [Baseline Defenses for Adversarial Attacks Against Aligned Language Models](https://mlsanity.com/paper/2309.00614/) (2023)
- [Visual Adversarial Examples Jailbreak Aligned Large Language Models](https://mlsanity.com/paper/2306.13213/) (2023)
- [Llama Guard 3 Vision: Safeguarding Human-AI Image Understanding Conversations](https://mlsanity.com/paper/2411.10414/) (2024)
- [AutoDAN: Interpretable Gradient-Based Adversarial Attacks on Large Language Models](https://mlsanity.com/paper/2310.15140/) (2023)
- [Exploring the Adversarial Vulnerabilities of Vision-Language-Action Models in Robotics](https://mlsanity.com/paper/2411.13587/) (2024)
- [PatchCleanser: Certifiably Robust Defense against Adversarial Patches for Any Image Classifier](https://mlsanity.com/paper/2108.09135/) (2021)
- [From Robustness to Explainability and Back Again](https://mlsanity.com/paper/2306.03048/) (2023)
- [When Robots Obey the Patch: Universal Transferable Patch Attacks on Vision-Language-Action Models](https://mlsanity.com/paper/2511.21192/) (2025)
- [Detecting Brittle Decisions for Free: Leveraging Margin Consistency in Deep Robust Classifiers](https://mlsanity.com/paper/2406.18451/) (2024)
- [A Manifold Two-Sample Test Study: Integral Probability Metric with Neural Networks](https://mlsanity.com/paper/2205.02043/) (2022)
- [Be Your Own Neighborhood: Detecting Adversarial Example by the Neighborhood Relations Built on Self-Supervised Learning](https://mlsanity.com/paper/2209.00005/) (2022)
- [Certifiable Black-Box Attacks with Randomized Adversarial Examples: Breaking Defenses with Provable Confidence](https://mlsanity.com/paper/2304.04343/) (2023)
- [Splitting the Difference on Adversarial Training](https://mlsanity.com/paper/2310.02480/) (2023)
- [Improving Adversarial Robustness via Joint Classification and Multiple Explicit Detection Classes](https://mlsanity.com/paper/2210.14410/) (2022)
- [Stratified Adversarial Robustness with Rejection](https://mlsanity.com/paper/2305.01139/) (2023)
- [Beyond Labeling Oracles: What does it mean to steal ML models?](https://mlsanity.com/paper/2310.01959/) (2023)
- [The Uncanny Valley: Exploring Adversarial Robustness from a Flatness Perspective](https://mlsanity.com/paper/2405.16918/) (2024)
- [Towards Robust Vision Transformer via Masked Adaptive Ensemble](https://mlsanity.com/paper/2407.15385/) (2024)
- [Publishing Efficient On-device Models Increases Adversarial Vulnerability](https://mlsanity.com/paper/2212.13700/) (2022)
- [Robust Proxy: Improving Adversarial Robustness by Robust Proxy Learning](https://mlsanity.com/paper/2306.15457/) (2023)
- [HoSNN: Adversarially-Robust Homeostatic Spiking Neural Networks with Adaptive Firing Thresholds](https://mlsanity.com/paper/2308.10373/) (2023)
- [PASA: Attack Agnostic Unsupervised Adversarial Detection using Prediction & Attribution Sensitivity Analysis](https://mlsanity.com/paper/2404.10789/) (2024)
- [Evaluation of Parameter-based Attacks against Embedded Neural Networks with Laser Injection](https://mlsanity.com/paper/2304.12876/) (2023)
- [Attacking Bayes: On the Adversarial Robustness of Bayesian Neural Networks](https://mlsanity.com/paper/2404.19640/) (2024)
- [Deciphering the Definition of Adversarial Robustness for post-hoc OOD Detectors](https://mlsanity.com/paper/2406.15104/) (2024)
- [A Comprehensive Assessment Benchmark for Rigorously Evaluating Deep Learning Image Classifiers](https://mlsanity.com/paper/2308.04137/) (2023)
- [Is Certifying ℓp Robustness Still Worthwhile?](https://mlsanity.com/paper/2310.09361/) (2023)
- [Fight Fire with Fire: Combating Adversarial Patch Attacks using Pattern-randomized Defensive Patches](https://mlsanity.com/paper/2311.06122/) (2023)
- [Breaking Boundaries: Balancing Performance and Robustness in Deep Wireless Traffic Forecasting](https://mlsanity.com/paper/2311.09790/) (2023)
- [SoK: Analyzing Adversarial Examples: A Framework to Study Adversary Knowledge](https://mlsanity.com/paper/2402.14937/) (2024)

## Abstract (arXiv)

Making classifiers robust to adversarial examples is hard. Thus, many defenses tackle the seemingly easier task of detecting perturbed inputs. We show a barrier towards this goal. We prove a general hardness reduction between detection and classification of adversarial examples: given a robust detector for attacks at distance ε (in some metric), we can build a similarly robust (but inefficient) classifier for attacks at distance ε/2. Our reduction is computationally inefficient, and thus cannot be used to build practical classifiers. Instead, it is a useful sanity check to test whether empirical detection results imply something much stronger than the authors presumably anticipated. To illustrate, we revisit 13 detector defenses. For 11/13 cases, we show that the claimed detection results would imply an inefficient classifier with robustness far beyond the state-of-the-art.