mlsanityData to 7 Oct 2026

Adversarial attacks and defenses in explainable artificial intelligence: A survey

arXiv 2306.06123 ↗First posted 6 Jun 2023cs.CR cs.AI cs.CVcited by 47

Not on the map. Why?

Abstract (arXiv)

Explainable artificial intelligence (XAI) methods are portrayed as a remedy for debugging and trusting statistical and deep learning models, as well as interpreting their predictions. However, recent advances in adversarial machine learning (AdvML) highlight the limitations and vulnerabilities of state-of-the-art explanation methods, putting their security and trustworthiness into question. The possibility of manipulating, fooling or fairwashing evidence of the model's reasoning has detrimental consequences when applied in high-stakes decision-making and knowledge discovery. This survey provides a comprehensive overview of research concerning adversarial attacks on explanations of machine learning models, as well as fairness metrics. We introduce a unified notation and taxonomy of methods facilitating a common ground for researchers and practitioners from the intersecting research fields of AdvML and XAI. We discuss how to defend against attacks and design robust interpretation methods. We contribute a list of existing insecurities in XAI and outline the emerging research directions in adversarial XAI (AdvXAI). Future work should address improving explanation methods and evaluation protocols to take into account the reported safety issues.

Key facts. Cited by 47 papers in the map (data through 7 Oct 2026). Builds on 122 of its 218 references that are in the map. First posted on arXiv 6 Jun 2023.

Builds on

122 of 218 references.

References · 122

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  2. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  3. 2023Segment Anythingcited by 8,158
  4. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  5. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  6. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,120
  7. 2013Intriguing properties of neural networkscited by 5,114
  8. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  9. 2017Axiomatic Attribution for Deep Networkscited by 3,350
  10. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  11. 2015Learning Deep Features for Discriminative Localizationcited by 2,430
  12. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,348
38 more
  1. 2016Equality of Opportunity in Supervised Learningcited by 2,000
  2. 2019A Survey on Bias and Fairness in Machine Learningcited by 1,911
  3. 2017Understanding Black-box Predictions via Influence Functionscited by 1,714
  4. 2017Learning Important Features Through Propagating Activation Differencescited by 1,454
  5. 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,309
  6. 2017Explanation in Artificial Intelligence: Insights from the Social Sciencescited by 1,201
  7. 2018Sanity Checks for Saliency Mapscited by 1,149
  8. 2017SmoothGrad: removing noise by adding noisecited by 1,097
  9. 2017BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chaincited by 1,052
  10. 2017Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoningcited by 1,012
  11. 2014Striving for Simplicity: The All Convolutional Netcited by 983
  12. 2018A Survey Of Methods For Explaining Black Box Modelscited by 970
  13. 2015The Limitations of Deep Learning in Adversarial Settingscited by 784
  14. 2014Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Imagescited by 774
  15. 2018This Looks Like That: Deep Learning for Interpretable Image Recognitioncited by 731
  16. 2017Evasion Attacks against Machine Learning at Test Timecited by 686
  17. 2015Distillation as a Defense to Adversarial Perturbations against Deep Neural Networkscited by 675
  18. 2022TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Secondcited by 643
  19. 2017One pixel attack for fooling deep neural networkscited by 579
  20. 2017Adversarial Patchcited by 514
  21. 2018Towards Robust Interpretability with Self-Explaining Neural Networkscited by 505
  22. 2017Interpretation of Neural Networks is Fragilecited by 443
  23. 2019Fooling LIME and SHAP: Adversarial Attacks on Post hoc Explanation Methodscited by 440
  24. 2017Wild Patterns: Ten Years After the Rise of Adversarial Machine Learningcited by 411
  25. 2017Algorithmic decision making and the cost of fairnesscited by 402
  26. 2018All Models are Wrong, but Many are Useful: Learning a Variable's Importance by Studying an Entire Class of Prediction Models Simultaneouslycited by 364
  27. 2017The (Un)reliability of saliency methodscited by 331
  28. 2019On the (In)fidelity and Sensitivity for Explanationscited by 309
  29. 2022From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AIcited by 299
  30. 2018Manipulating and Measuring Model Interpretabilitycited by 256
  31. 2017Real Time Image Saliency for Black Box Classifierscited by 246
  32. 2017On Detecting Adversarial Perturbationscited by 232
  33. 2019Explanations can be manipulated and geometry is to blamecited by 215
  34. 2019Explaining individual predictions when features are dependent: More accurate approximations to Shapley valuescited by 205
  35. 2019Explainable Reinforcement Learning Through a Causal Lenscited by 177
  36. 2022Quantus: An Explainable AI Toolkit for Responsible Evaluation of Neural Network Explanations and Beyondcited by 171
  37. 2018Abduction-Based Explanations for Machine Learning Modelscited by 159
  38. 2014Towards Deep Neural Network Architectures Robust to Adversarial Examplescited by 146

50 most cited of 122 All 122 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 22; 2025: 13; 2026: 9.

47 papers.

Sort:

By area:

  1. 2024Adversarial Attacks and Defenses on Text-to-Image Diffusion Models: A Surveycited by 34
  2. 2023The role of causality in explainable artificial intelligencecited by 17
  3. 2024XAI-Based Detection of Adversarial Attacks on Deepfake Detectorscited by 9
  4. 2025Investigating the Duality of Interpretability and Explainability in Machine Learningcited by 8
  5. 2024Provably Better Explanations with Optimized Aggregation of Feature Attributionscited by 6
  6. 2023Adversarial Attacks and Defenses on 3D Point Cloud Classification: A Surveycited by 5
  7. 2024X Hacking: The Threat of Misguided AutoMLcited by 5
  8. 2024Why You Should Not Trust Interpretations in Machine Learning: Adversarial Attacks on Partial Dependence Plotscited by 5
  9. 2024Explainable Graph Neural Networks Under Firecited by 5
  10. 2024When Can You Trust Your Explanations? A Robustness Analysis on Feature Importancescited by 5
  11. 2023Survey on AI Ethics: A Socio-technical Perspectivecited by 4
  12. 2024False Sense of Security in Explainable Artificial Intelligence (XAI)cited by 4
35 more
  1. 2024Algebraic Adversarial Attacks on Integrated Gradientscited by 4
  2. 2024Fooling SHAP with Output Shuffling Attackscited by 4
  3. 2025Pixel-level Certified Explanations via Randomized Smoothingcited by 4
  4. 2024The Effect of Data Poisoning on Counterfactual Explanationscited by 3
  5. 2024Self and Cross-Model Distillation for LLMs: Effective Methods for Refusal Pattern Alignmentcited by 3
  6. 2025Birds look like cars: Adversarial analysis of intrinsically interpretable deep learningcited by 3
  7. 2025Model Science: getting serious about verification, explanation and control of AI systemscited by 3
  8. 2024On the Robustness of Global Feature Effect Explanationscited by 2
  9. 2024Joint Universal Adversarial Perturbations with Interpretationscited by 2
  10. 2024Faithfulness and the Notion of Adversarial Sensitivity in NLP Explanationscited by 2
  11. 2024CausAdv: A Causal-based Framework for Detecting Adversarial Examplescited by 2
  12. 2025Factor Graph-based Interpretable Neural Networkscited by 2
  13. 2025A Survey on Interpretability in Visual Recognitioncited by 2
  14. 2025Enhancing Security in Deep Reinforcement Learning: A Comprehensive Survey on Adversarial Attacks and Defensescited by 2
  15. 2025Improving Perturbation-based Explanations by Understanding the Role of Uncertainty Calibrationcited by 2
  16. 2024XAI-CF -- Examining the Role of Explainable Artificial Intelligence in Cyber Forensicscited by 1
  17. 2025Explainable but Vulnerable: Adversarial Attacks on XAI Explanation in Cybersecurity Applicationscited by 1
  18. 2026Right Predictions, Misleading Explanations: On the Vulnerability of Vision-Language Model Explanationscited by 1
  19. 2024Towards White Box Deep Learningcited by 0
  20. 2024Revealing Vulnerabilities of Neural Networks in Parameter Learning and Defense Against Explanation-Aware Backdoorscited by 0
  21. 2024Towards Understanding the Influence of Training Samples on Explanationscited by 0
  22. 2024Top K Enhanced Reinforcement Learning Attacks on Heterogeneous Graph Node Classificationcited by 0
  23. 2024Advancing NLP Security by Leveraging LLMs as Adversarial Enginescited by 0
  24. 2025Algebraic Adversarial Attacks on Explainability Modelscited by 0
  25. 2025Autonomous Cyber Resilience via a Co-Evolutionary Arms Race within a Fortified Digital Twin Sandboxcited by 0
  26. 2025Pulling Back the Curtain on Deep Networkscited by 0
  27. 2025Actionable and diverse counterfactual explanations incorporating domain knowledge and plausibility constraintscited by 0
  28. 2026On the Definition and Detection of Cherry-Picking in Counterfactual Explanationscited by 0
  29. 2026Universal Adversarial Purification with DDIM Metric Loss for Stable Diffusioncited by 0
  30. 2026Evasion of IoT Malware Detection via Dummy Code Injectioncited by 0
  31. 2026SCAN: Visual Explanations with Self-Confidence and Analysis Networkscited by 0
  32. 2026Uncertainty Gating for Cost-Aware Explainable Artificial Intelligencecited by 0
  33. 2026eDySec: A Deep Learning-based Explainable Dynamic Analysis Framework for Detecting Malicious Packages in PyPI Ecosystemcited by 0
  34. 2026JECA^2: Judgment-Explanation Consistent Adversarial Attack against Forensic Vision-Language Modelscited by 0
  35. 2026Are the High-weight Neurons the Important Ones in Image Classification Neural Networks?cited by 0