# When Explainability Meets Adversarial Learning: Detecting Adversarial Examples using SHAP Signatures

arXiv: https://arxiv.org/abs/1909.03418 · HTML: https://mlsanity.com/paper/1909.03418/

## Key facts

- Cited by 40 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Sep 2019.

## Built on by (most cited first, 30 of 40)

- [On the Complexity of SHAP-Score-Based Explanations: Tractability via Knowledge Compilation and Non-Approximability Results](https://mlsanity.com/paper/2104.08015/) (2021)
- ["That Is a Suspicious Reaction!": Interpreting Logits Variation to Detect NLP Adversarial Attacks](https://mlsanity.com/paper/2204.04636/) (2022)
- [Identifying regions of importance in wall-bounded turbulence through explainable deep learning](https://mlsanity.com/paper/2302.01250/) (2023)
- [Foiling Explanations in Deep Neural Networks](https://mlsanity.com/paper/2211.14860/) (2022)
- [Robust Adversarial Attacks Detection based on Explainable Deep Reinforcement Learning For UAV Guidance and Planning](https://mlsanity.com/paper/2206.02670/) (2022)
- [Generalizing Adversarial Explanations with Grad-CAM](https://mlsanity.com/paper/2204.05427/) (2022)
- [A Survey on XAI for 5G and Beyond Security: Technical Aspects, Challenges and Research Directions](https://mlsanity.com/paper/2204.12822/) (2022)
- [Explainable Artificial Intelligence (XAI) for Internet of Things: A Survey](https://mlsanity.com/paper/2206.04800/) (2022)
- [Impacts and Risk of Generative AI Technology on Cyber Defense](https://mlsanity.com/paper/2306.13033/) (2023)
- [Explanation-Guided Diagnosis of Machine Learning Evasion Attacks](https://mlsanity.com/paper/2106.15820/) (2021)
- [A Review of Explainable Artificial Intelligence in Manufacturing](https://mlsanity.com/paper/2107.02295/) (2021)
- [Improving Interpretability via Regularization of Neural Activation Sensitivity](https://mlsanity.com/paper/2211.08686/) (2022)
- [Explainable Artificial Intelligence (XAI) for Malware Analysis: A Survey of Techniques, Applications, and Open Challenges](https://mlsanity.com/paper/2409.13723/) (2024)
- [Backdooring Explainable Machine Learning](https://mlsanity.com/paper/2204.09498/) (2022)
- [Exploring Adversarial Attacks and Defenses in Vision Transformers trained with DINO](https://mlsanity.com/paper/2206.06761/) (2022)
- [Relating tSNE and UMAP to Classical Dimensionality Reduction](https://mlsanity.com/paper/2306.11898/) (2023)
- [Trustworthy Actionable Perturbations](https://mlsanity.com/paper/2405.11195/) (2024)
- [On Gradient-like Explanation under a Black-box Setting: When Black-box Explanations Become as Good as White-box](https://mlsanity.com/paper/2308.09381/) (2023)
- [Securing Virtual Reality Experiences: Unveiling and Tackling Cybersickness Attacks with Explainable AI](https://mlsanity.com/paper/2503.13419/) (2025)
- [STARdom: an architecture for trusted and secure human-centered manufacturing systems](https://mlsanity.com/paper/2104.00983/) (2021)
- [Detection of Adversarial Physical Attacks in Time-Series Image Data](https://mlsanity.com/paper/2304.13919/) (2023)
- [X-Detect: Explainable Adversarial Patch Detection for Object Detectors in Retail](https://mlsanity.com/paper/2306.08422/) (2023)
- [Interpreting Outliers in Time Series Data through Decoding Autoencoder](https://mlsanity.com/paper/2409.01713/) (2024)
- [Explainable but Vulnerable: Adversarial Attacks on XAI Explanation in Cybersecurity Applications](https://mlsanity.com/paper/2510.03623/) (2025)
- [Towards an Explanation Space to Align Humans and Explainable-AI Teamwork](https://mlsanity.com/paper/2106.01503/) (2021)
- [Unsupervised Detection of Adversarial Examples with Model Explanations](https://mlsanity.com/paper/2107.10480/) (2021)
- [Detecting Adversaries, yet Faltering to Noise? Leveraging Conditional Variational AutoEncoders for Adversary Detection in the Presence of Noisy Images](https://mlsanity.com/paper/2111.15518/) (2021)
- [Btech thesis report on adversarial attack detection and purification of adverserially attacked images](https://mlsanity.com/paper/2205.07859/) (2022)
- [A Melting Pot of Evolution and Learning](https://mlsanity.com/paper/2306.04971/) (2023)
- [XFedHunter: An Explainable Federated Learning Framework for Advanced Persistent Threat Detection in SDN](https://mlsanity.com/paper/2309.08485/) (2023)

## Abstract (arXiv)

State-of-the-art deep neural networks (DNNs) are highly effective in solving many complex real-world problems. However, these models are vulnerable to adversarial perturbation attacks, and despite the plethora of research in this domain, to this day, adversaries still have the upper hand in the cat and mouse game of adversarial example generation methods vs. detection and prevention methods. In this research, we present a novel detection method that uses Shapley Additive Explanations (SHAP) values computed for the internal layers of a DNN classifier to discriminate between normal and adversarial inputs. We evaluate our method by building an extensive dataset of adversarial examples over the popular CIFAR-10 and MNIST datasets, and training a neural network-based detector to distinguish between normal and adversarial inputs. We evaluate our detector against adversarial examples generated by diverse state-of-the-art attacks and demonstrate its high detection accuracy and strong generalization ability to adversarial inputs generated with different attack methods.