mlsanityData to 14 Oct 2026

Revealing Vulnerabilities of Neural Networks in Parameter Learning and Defense Against Explanation-Aware Backdoors

arXiv 2403.16569 ↗First posted 25 Mar 2024cs.LG cs.CR cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Explainable Artificial Intelligence (XAI) strategies play a crucial part in increasing the understanding and trustworthiness of neural networks. Nonetheless, these techniques could potentially generate misleading explanations. Blinding attacks can drastically alter a machine learning algorithm's prediction and explanation, providing misleading information by adding visually unnoticeable artifacts into the input, while maintaining the model's accuracy. It poses a serious challenge in ensuring the reliability of XAI methods. To ensure the reliability of XAI methods poses a real challenge, we leverage statistical analysis to highlight the changes in CNN weights within a CNN following blinding attacks. We introduce a method specifically designed to limit the effectiveness of such attacks during the evaluation phase, avoiding the need for extra training. The method we suggest defences against most modern explanation-aware adversarial attacks, achieving an approximate decrease of ~99% in the Attack Success Rate (ASR) and a ~91% reduction in the Mean Square Error (MSE) between the original explanation and the defended (post-attack) explanation across three unique types of attacks.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 29 of its 46 references that are in the map. First posted on arXiv 25 Mar 2024.

Builds on

29 of 46 references.

References · 29

  1. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  2. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
  3. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,733
  4. 2013Visualizing and Understanding Convolutional Networkscited by 2,290
  5. 2019Similarity of Neural Network Representations Revisitedcited by 1,755
  6. 2018Sanity Checks for Saliency Mapscited by 1,156
  7. 2019Unmasking Clever Hans Predictors and Assessing What Machines Really Learncited by 365
  8. 2019Explanations can be manipulated and geometry is to blamecited by 216
  9. 2022From Attribution Maps to Human-Understandable Explanations through Concept Relevance Propagationcited by 141
  10. 2023Explainable Artificial Intelligence (XAI) 2.0: A Manifesto of Open Challenges and Interdisciplinary Research Directionscited by 125
  11. 2019Fooling Neural Network Interpretations via Adversarial Model Manipulationcited by 112
  12. 2020Fairwashing Explanations with Off-Manifold Detergentcited by 72
17 more
  1. 2023Adversarial attacks and defenses in explainable artificial intelligence: A surveycited by 48
  2. 2019Robust Attribution Regularizationcited by 47
  3. 2020Robust and Stable Black Box Explanationscited by 47
  4. 2020Towards Robust Explanations for Deep Neural Networkscited by 43
  5. 2018Explicit Bias Discovery in Visual Question Answering Modelscited by 42
  6. 2019When Explainability Meets Adversarial Learning: Detecting Adversarial Examples using SHAP Signaturescited by 40
  7. 2020Batch Normalization Increases Adversarial Vulnerability and Decreases Adversarial Transferability: A Non-Robust Feature Perspectivecited by 35
  8. 2022SAFARI: Versatile and Efficient Evaluations for Robustness of Interpretabilitycited by 28
  9. 2022Removing Batch Normalization Boosts Adversarial Trainingcited by 24
  10. 2022Robust Explanation Constraints for Neural Networkscited by 24
  11. 2020A simple defense against adversarial attacks on heatmap explanationscited by 23
  12. 2022Shortcomings of Top-Down Randomization-Based Sanity Checks for Evaluations of Deep Neural Network Explanationscited by 22
  13. 2023Defending Against Patch-based Backdoor Attacks on Self-Supervised Learningcited by 20
  14. 2022Deconfounded Representation Similarity for Comparison of Neural Networkscited by 19
  15. 2022"Is your explanation stable?": A Robustness Evaluation Framework for Feature Attributioncited by 7
  16. 2021Defense Against Explanation Manipulationcited by 6
  17. 2022Towards More Robust Interpretation via Local Gradient Alignmentcited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Explainable but Vulnerable: Adversarial Attacks on XAI Explanation in Cybersecurity Applications
  2. 2022Backdooring Explainable Machine Learning
  3. 2022Foiling Explanations in Deep Neural Networks
  4. 2024XSub: Explanation-Driven Adversarial Attack against Blackbox Classifiers via Feature Substitution
  5. 2026Right Predictions, Misleading Explanations: On the Vulnerability of Vision-Language Model Explanations
  6. 2025Illuminating the Black Box: Real-Time Monitoring of Backdoor Unlearning in CNNs via Explainable AI ↗
  7. 2023AUTOLYCUS: Exploiting Explainable AI (XAI) for Model Extraction Attacks against Interpretable Models
  8. 2025Birds look like cars: Adversarial analysis of intrinsically interpretable deep learning