# Adversarial attacks and defenses in explainable artificial intelligence: A survey

arXiv: https://arxiv.org/abs/2306.06123 · HTML: https://mlsanity.com/paper/2306.06123/

## Key facts

- Cited by 48 papers in the map (data through 14 Oct 2026).
- Builds on 122 of its 218 references that are in the map.
- First posted on arXiv 6 Jun 2023.

## Builds on (most cited first, 30 of 122)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Sparks of Artificial General Intelligence: Early experiments with GPT-4](https://mlsanity.com/paper/2303.12712/) (2023)
- [Learning Deep Features for Discriminative Localization](https://mlsanity.com/paper/1512.04150/) (2015)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [Equality of Opportunity in Supervised Learning](https://mlsanity.com/paper/1610.02413/) (2016)
- [A Survey on Bias and Fairness in Machine Learning](https://mlsanity.com/paper/1908.09635/) (2019)
- [Understanding Black-box Predictions via Influence Functions](https://mlsanity.com/paper/1703.04730/) (2017)
- [Learning Important Features Through Propagating Activation Differences](https://mlsanity.com/paper/1704.02685/) (2017)
- [Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)](https://mlsanity.com/paper/1711.11279/) (2017)
- [Explanation in Artificial Intelligence: Insights from the Social Sciences](https://mlsanity.com/paper/1706.07269/) (2017)
- [Sanity Checks for Saliency Maps](https://mlsanity.com/paper/1810.03292/) (2018)
- [SmoothGrad: removing noise by adding noise](https://mlsanity.com/paper/1706.03825/) (2017)
- [BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain](https://mlsanity.com/paper/1708.06733/) (2017)
- [Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning](https://mlsanity.com/paper/1712.05526/) (2017)
- [Striving for Simplicity: The All Convolutional Net](https://mlsanity.com/paper/1412.6806/) (2014)
- [A Survey Of Methods For Explaining Black Box Models](https://mlsanity.com/paper/1802.01933/) (2018)
- [The Limitations of Deep Learning in Adversarial Settings](https://mlsanity.com/paper/1511.07528/) (2015)
- [Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Images](https://mlsanity.com/paper/1412.1897/) (2014)
- [This Looks Like That: Deep Learning for Interpretable Image Recognition](https://mlsanity.com/paper/1806.10574/) (2018)
- [Evasion Attacks against Machine Learning at Test Time](https://mlsanity.com/paper/1708.06131/) (2017)
- [Distillation as a Defense to Adversarial Perturbations against Deep Neural Networks](https://mlsanity.com/paper/1511.04508/) (2015)
- [TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Second](https://mlsanity.com/paper/2207.01848/) (2022)

## Built on by (most cited first, 30 of 48)

- [Adversarial Attacks and Defenses on Text-to-Image Diffusion Models: A Survey](https://mlsanity.com/paper/2407.15861/) (2024)
- [The role of causality in explainable artificial intelligence](https://mlsanity.com/paper/2309.09901/) (2023)
- [XAI-Based Detection of Adversarial Attacks on Deepfake Detectors](https://mlsanity.com/paper/2403.02955/) (2024)
- [Investigating the Duality of Interpretability and Explainability in Machine Learning](https://mlsanity.com/paper/2503.21356/) (2025)
- [Provably Better Explanations with Optimized Aggregation of Feature Attributions](https://mlsanity.com/paper/2406.05090/) (2024)
- [Adversarial Attacks and Defenses on 3D Point Cloud Classification: A Survey](https://mlsanity.com/paper/2307.00309/) (2023)
- [X Hacking: The Threat of Misguided AutoML](https://mlsanity.com/paper/2401.08513/) (2024)
- [Why You Should Not Trust Interpretations in Machine Learning: Adversarial Attacks on Partial Dependence Plots](https://mlsanity.com/paper/2404.18702/) (2024)
- [Explainable Graph Neural Networks Under Fire](https://mlsanity.com/paper/2406.06417/) (2024)
- [When Can You Trust Your Explanations? A Robustness Analysis on Feature Importances](https://mlsanity.com/paper/2406.14349/) (2024)
- [Survey on AI Ethics: A Socio-technical Perspective](https://mlsanity.com/paper/2311.17228/) (2023)
- [False Sense of Security in Explainable Artificial Intelligence (XAI)](https://mlsanity.com/paper/2405.03820/) (2024)
- [Algebraic Adversarial Attacks on Integrated Gradients](https://mlsanity.com/paper/2407.16233/) (2024)
- [Fooling SHAP with Output Shuffling Attacks](https://mlsanity.com/paper/2408.06509/) (2024)
- [Pixel-level Certified Explanations via Randomized Smoothing](https://mlsanity.com/paper/2506.15499/) (2025)
- [The Effect of Data Poisoning on Counterfactual Explanations](https://mlsanity.com/paper/2402.08290/) (2024)
- [Self and Cross-Model Distillation for LLMs: Effective Methods for Refusal Pattern Alignment](https://mlsanity.com/paper/2406.11285/) (2024)
- [Birds look like cars: Adversarial analysis of intrinsically interpretable deep learning](https://mlsanity.com/paper/2503.08636/) (2025)
- [Model Science: getting serious about verification, explanation and control of AI systems](https://mlsanity.com/paper/2508.20040/) (2025)
- [Improving Perturbation-based Explanations by Understanding the Role of Uncertainty Calibration](https://mlsanity.com/paper/2511.10439/) (2025)
- [On the Robustness of Global Feature Effect Explanations](https://mlsanity.com/paper/2406.09069/) (2024)
- [Joint Universal Adversarial Perturbations with Interpretations](https://mlsanity.com/paper/2408.01715/) (2024)
- [Faithfulness and the Notion of Adversarial Sensitivity in NLP Explanations](https://mlsanity.com/paper/2409.17774/) (2024)
- [CausAdv: A Causal-based Framework for Detecting Adversarial Examples](https://mlsanity.com/paper/2411.00839/) (2024)
- [Factor Graph-based Interpretable Neural Networks](https://mlsanity.com/paper/2502.14572/) (2025)
- [A Survey on Interpretability in Visual Recognition](https://mlsanity.com/paper/2507.11099/) (2025)
- [Enhancing Security in Deep Reinforcement Learning: A Comprehensive Survey on Adversarial Attacks and Defenses](https://mlsanity.com/paper/2510.20314/) (2025)
- [XAI-CF -- Examining the Role of Explainable Artificial Intelligence in Cyber Forensics](https://mlsanity.com/paper/2402.02452/) (2024)
- [Explainable but Vulnerable: Adversarial Attacks on XAI Explanation in Cybersecurity Applications](https://mlsanity.com/paper/2510.03623/) (2025)
- [BadTemplate: A Training-Free Backdoor Attack via Chat Template Against Large Language Models](https://mlsanity.com/paper/2602.05401/) (2026)

## Abstract (arXiv)

Explainable artificial intelligence (XAI) methods are portrayed as a remedy for debugging and trusting statistical and deep learning models, as well as interpreting their predictions. However, recent advances in adversarial machine learning (AdvML) highlight the limitations and vulnerabilities of state-of-the-art explanation methods, putting their security and trustworthiness into question. The possibility of manipulating, fooling or fairwashing evidence of the model's reasoning has detrimental consequences when applied in high-stakes decision-making and knowledge discovery. This survey provides a comprehensive overview of research concerning adversarial attacks on explanations of machine learning models, as well as fairness metrics. We introduce a unified notation and taxonomy of methods facilitating a common ground for researchers and practitioners from the intersecting research fields of AdvML and XAI. We discuss how to defend against attacks and design robust interpretation methods. We contribute a list of existing insecurities in XAI and outline the emerging research directions in adversarial XAI (AdvXAI). Future work should address improving explanation methods and evaluation protocols to take into account the reported safety issues.