mlsanityData to 7 Oct 2026

Explanations can be manipulated and geometry is to blame

arXiv 1906.07983 ↗First posted 19 Jun 2019stat.ML cs.CR cs.LGcited by 215

Not on the map. Why?

Abstract (arXiv)

Explanation methods aim to make neural networks more trustworthy and interpretable. In this paper, we demonstrate a property of explanation methods which is disconcerting for both of these purposes. Namely, we show that explanations can be manipulated arbitrarily by applying visually hardly perceptible perturbations to the input that keep the network's output approximately constant. We establish theoretically that this phenomenon can be related to certain geometrical properties of neural networks. This allows us to derive an upper bound on the susceptibility of explanations to manipulations. Based on this result, we propose effective mechanisms to enhance the robustness of explanations.

Key facts. Cited by 215 papers in the map (data through 7 Oct 2026). First posted on arXiv 19 Jun 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 39; 2022: 56; 2023: 36; 2024: 40; 2025: 25; 2026: 19.

215 papers.

Sort:

By area:

  1. 2022From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AIcited by 299
  2. 2022Extremely Simple Activation Shaping for Out-of-Distribution Detectioncited by 212
  3. 2022The Disagreement Problem in Explainable Machine Learning: A Practitioner's Perspectivecited by 156
  4. 2021Anti-Adversarially Manipulated Attributions for Weakly and Semi-Supervised Semantic Segmentationcited by 134
  5. 2022Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networkscited by 126
  6. 2023Faithfulness Tests for Natural Language Explanationscited by 107
  7. 2021Guided Integrated Gradients: An Adaptive Path Method for Removing Noisecited by 102
  8. 2022Which Explanation Should I Choose? A Function Approximation Perspective to Characterizing Post Hoc Explanationscited by 77
  9. 2022GlanceNets: Interpretabile, Leak-proof Concept-based Modelscited by 73
  10. 2021Synthetic Benchmarks for Scientific Research in Explainable Machine Learningcited by 59
  11. 2022Post hoc Explanations may be Ineffective for Detecting Unknown Spurious Correlationcited by 59
  12. 2021Do Input Gradients Highlight Discriminative Features?cited by 52
38 more
  1. 2022Rethinking Explainability as a Dialogue: A Practitioner's Perspectivecited by 52
  2. 2022Rethinking Stability for Attribution-based Explanationscited by 52
  3. 2022Concept-level Debugging of Part-Prototype Networkscited by 52
  4. 2022Leveraging Explanations in Interactive Machine Learning: An Overviewcited by 52
  5. 2023Sequential Integrated Gradients: a simple but effective method for explaining language modelscited by 52
  6. 2021Robust Explainability: A Tutorial on Gradient-Based Attribution Methods for Deep Neural Networkscited by 48
  7. 2022Beyond Explaining: Opportunities and Challenges of XAI-Based Model Improvementcited by 48
  8. 2021Towards the Unification and Robustness of Perturbation and Gradient Based Explanationscited by 47
  9. 2023Adversarial attacks and defenses in explainable artificial intelligence: A surveycited by 47
  10. 2022Spurious Features Everywhere -- Large-Scale Detection of Harmful Spurious Features in ImageNetcited by 41
  11. 2022Fairness via Explanation Quality: Evaluating Disparities in the Quality of Post hoc Explanationscited by 40
  12. 2021A Survey on the Robustness of Feature Importance and Counterfactual Explanationscited by 38
  13. 2021Consistent Counterfactuals for Deep Modelscited by 37
  14. 2023The Meta-Evaluation Problem in Explainable AI: Identifying Reliable Estimators with MetaQuantuscited by 37
  15. 2021Perturbing Inputs for Fragile Interpretations in Deep Natural Language Processingcited by 30
  16. 2021Toward Explainable AI for Regression Modelscited by 30
  17. 2022Sparse Visual Counterfactual Explanations in Image Spacecited by 30
  18. 2023Towards Trustworthy and Aligned Machine Learning: A Data-centric Survey with Causality Perspectivescited by 30
  19. 2022Diffeomorphic Counterfactuals with Generative Modelscited by 29
  20. 2021NoiseGrad: Enhancing Explanations by Introducing Stochasticity to Model Weightscited by 28
  21. 2022SAFARI: Versatile and Efficient Evaluations for Robustness of Interpretabilitycited by 28
  22. 2024Decoupling Pixel Flipping and Occlusion Strategy for Consistent XAI Benchmarkscited by 28
  23. 2022SEAT: Stable and Explainable Attentioncited by 25
  24. 2022Disentangled Explanations of Neural Network Predictions by Finding Relevant Subspacescited by 25
  25. 2021Selective Ensembles for Consistent Predictionscited by 23
  26. 2022Robust Explanation Constraints for Neural Networkscited by 23
  27. 2021Best of both worlds: local and global explanations with human-understandable conceptscited by 22
  28. 2022Understanding and Enhancing Robustness of Concept-based Modelscited by 22
  29. 2021Robust Models Are More Interpretable Because Attributions Look Normalcited by 21
  30. 2021Characterizing the risk of fairwashingcited by 21
  31. 2022What Makes a Good Explanation?: A Harmonized View of Properties of Explanationscited by 21
  32. 2024Manifold Integrated Gradients: Riemannian Geometry for Feature Attributioncited by 21
  33. 2022Attribution-based Explanations that Provide Recourse Cannot be Robustcited by 20
  34. 2022SoK: Explainable Machine Learning for Computer Security Applicationscited by 20
  35. 2022Efficiently Training Low-Curvature Neural Networkscited by 19
  36. 2023Explainable Brain Age Prediction using coVariance Neural Networkscited by 19
  37. 2021Explaining Bayesian Neural Networkscited by 18
  38. 2023On the Robustness of Removal-Based Feature Attributionscited by 18

50 most cited of 215 All 215 (JSON)