# Explanation-Guided Diagnosis of Machine Learning Evasion Attacks

arXiv: https://arxiv.org/abs/2106.15820 · HTML: https://mlsanity.com/paper/2106.15820/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 60 references that are in the map.
- First posted on arXiv 30 Jun 2021.

## Builds on (most cited first, 30 of 35)

- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Towards Evaluating the Robustness of Neural Networks](https://mlsanity.com/paper/1608.04644/) (2016)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [Learning Important Features Through Propagating Activation Differences](https://mlsanity.com/paper/1704.02685/) (2017)
- [SmoothGrad: removing noise by adding noise](https://mlsanity.com/paper/1706.03825/) (2017)
- [Striving for Simplicity: The All Convolutional Net](https://mlsanity.com/paper/1412.6806/) (2014)
- [Adversarial Machine Learning at Scale](https://mlsanity.com/paper/1611.01236/) (2016)
- [Practical Black-Box Attacks against Machine Learning](https://mlsanity.com/paper/1602.02697/) (2016)
- [Interpretable Explanations of Black Boxes by Meaningful Perturbation](https://mlsanity.com/paper/1704.03296/) (2017)
- [Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples](https://mlsanity.com/paper/1605.07277/) (2016)
- [Interpretation of Neural Networks is Fragile](https://mlsanity.com/paper/1710.10547/) (2017)
- [Wild Patterns: Ten Years After the Rise of Adversarial Machine Learning](https://mlsanity.com/paper/1712.03141/) (2017)
- [Deep Reinforcement Learning framework for Autonomous Driving](https://mlsanity.com/paper/1704.02532/) (2017)
- [Understanding Neural Networks through Representation Erasure](https://mlsanity.com/paper/1612.08220/) (2016)
- [Local Rule-Based Explanations of Black Box Decision Systems](https://mlsanity.com/paper/1805.10820/) (2018)
- [EMBER: An Open Dataset for Training Static PE Malware Machine Learning Models](https://mlsanity.com/paper/1804.04637/) (2018)
- [Malware Detection by Eating a Whole EXE](https://mlsanity.com/paper/1710.09435/) (2017)
- [Fooling Neural Network Interpretations via Adversarial Model Manipulation](https://mlsanity.com/paper/1902.02041/) (2019)
- [Generating Adversarial Malware Examples for Black-Box Attacks Based on GAN](https://mlsanity.com/paper/1702.05983/) (2017)
- [Interpretable Deep Learning under Fire](https://mlsanity.com/paper/1812.00891/) (2018)
- [Adversarial Malware Binaries: Evading Deep Learning for Malware Detection in Executables](https://mlsanity.com/paper/1803.04173/) (2018)
- [Learning to Evade Static PE Machine Learning Malware Models via Reinforcement Learning](https://mlsanity.com/paper/1801.08917/) (2018)
- [Yes, Machine Learning Can Be More Secure! A Case Study on Android Malware Detection](https://mlsanity.com/paper/1704.08996/) (2017)
- [Exploring Adversarial Examples in Malware Detection](https://mlsanity.com/paper/1810.08280/) (2018)
- [When Explainability Meets Adversarial Learning: Detecting Adversarial Examples using SHAP Signatures](https://mlsanity.com/paper/1909.03418/) (2019)
- [Evaluating Explanation Methods for Deep Learning in Security](https://mlsanity.com/paper/1906.02108/) (2019)
- [Adversarial Deep Learning for Robust Detection of Binary Encoded Malware](https://mlsanity.com/paper/1801.02950/) (2018)

## Built on by (most cited first, 6 of 6)

- [The Role of Machine Learning in Cybersecurity](https://mlsanity.com/paper/2206.09707/) (2022)
- [EG-Booster: Explanation-Guided Booster of ML Evasion Attacks](https://mlsanity.com/paper/2108.13930/) (2021)
- [Concept-based Adversarial Attacks: Tricking Humans and Classifiers Alike](https://mlsanity.com/paper/2203.10166/) (2022)
- [Privacy Preservation in Artificial Intelligence and Extended Reality (AI-XR) Metaverses: A Survey](https://mlsanity.com/paper/2310.10665/) (2023)
- [Systems-Theoretic and Data-Driven Security Analysis in ML-enabled Medical Devices](https://mlsanity.com/paper/2506.15028/) (2025)
- [DeepProv: Behavioral Characterization and Repair of Neural Networks via Inference Provenance Graph Analysis](https://mlsanity.com/paper/2509.26562/) (2025)

## Abstract (arXiv)

Machine Learning (ML) models are susceptible to evasion attacks. Evasion accuracy is typically assessed using aggregate evasion rate, and it is an open question whether aggregate evasion rate enables feature-level diagnosis on the effect of adversarial perturbations on evasive predictions. In this paper, we introduce a novel framework that harnesses explainable ML methods to guide high-fidelity assessment of ML evasion attacks. Our framework enables explanation-guided correlation analysis between pre-evasion perturbations and post-evasion explanations. Towards systematic assessment of ML evasion attacks, we propose and evaluate a novel suite of model-agnostic metrics for sample-level and dataset-level correlation analysis. Using malware and image classifiers, we conduct comprehensive evaluations across diverse model architectures and complementary feature representations. Our explanation-guided correlation analysis reveals correlation gaps between adversarial samples and the corresponding perturbations performed on them. Using a case study on explanation-guided evasion, we show the broader usage of our methodology for assessing robustness of ML models.