MEAD: A Multi-Armed Approach for Evaluation of Adversarial Examples Detectors
Not on the map. Why?
Abstract (arXiv)
Detection of adversarial examples has been a hot topic in the last years due to its importance for safely deploying machine learning algorithms in critical applications. However, the detection methods are generally validated by assuming a single implicitly known attack strategy, which does not necessarily account for real-life threats. Indeed, this can lead to an overoptimistic assessment of the detectors' performance and may induce some bias in the comparison between competing detection schemes. We propose a novel multi-armed framework, called MEAD, for evaluating detectors based on several attack strategies to overcome this limitation. Among them, we make use of three new objectives to generate attacks. The proposed performance metric is based on the worst-case scenario: detection is successful if and only if all different attacks are correctly recognized. Empirically, we show the effectiveness of our approach. Moreover, the poor performance obtained for state-of-the-art detectors opens a new exciting line of research.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 28 of its 38 references that are in the map. First posted on arXiv 30 Jun 2022.
Builds on
28 of 38 references.
References · 28
- 2014Explaining and Harnessing Adversarial Examplescited by 7,287
- 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
- 2013Intriguing properties of neural networkscited by 5,114
- 2016Towards Evaluating the Robustness of Neural Networkscited by 2,976
- 2016Adversarial examples in the physical worldcited by 1,762
- 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,510
- 2015DeepFool: a simple and accurate method to fool deep neural networkscited by 1,315
- 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,259
- 2019Square Attack: a query-efficient black-box adversarial attack via random searchcited by 682
- 2020RobustBench: a standardized adversarial robustness benchmarkcited by 659
- 2017Feature Squeezing: Detecting Adversarial Examples in Deep Neural Networkscited by 484
- 2019Minimally distorted Adversarial Examples with a Fast Adaptive Boundary Attackcited by 333
16 more
- 2019HopSkipJumpAttack: A Query-Efficient Decision-Based Attackcited by 328
- 2018Characterizing Adversarial Subspaces Using Local Intrinsic Dimensionalitycited by 280
- 2017MagNet: a Two-Pronged Defense against Adversarial Examplescited by 268
- 2017Detecting Adversarial Samples from Artifactscited by 244
- 2017On Detecting Adversarial Perturbationscited by 232
- 2017On the (Statistical) Detection of Adversarial Examplescited by 180
- 2016Improving the Robustness of Deep Neural Networks via Stability Trainingcited by 156
- 2016Adversarial Examples Detection in Deep Networks with Convolutional Filter Statisticscited by 77
- 2017SafetyNet: Detecting and Rejecting Adversarial Examples Robustlycited by 76
- 2021DOCTOR: A Simple Method for Detecting Misclassification Errorscited by 52
- 2021Adversarial Example Detection for DNN Models: A Review and Experimental Comparisoncited by 51
- 2017Detecting Adversarial Image Examples in Deep Networks with Adaptive Noise Reductioncited by 49
- 2018Attacks Meet Interpretability: Attribute-steered Detection of Adversarial Samplescited by 38
- 2019Deep Neural Rejection against Adversarial Examplescited by 31
- 2018Distributionally Adversarial Attackcited by 24
- 2021Revisiting Model's Uncertainty and Confidences for Adversarial Example Detectioncited by 9
Built on by
Citing papers per year: 2023: 1; 2024: 1.
2 papers.