mlsanityData to 14 Oct 2026

When Explanations Lie: Why Many Modified BP Attributions Fail

arXiv 1912.09818 ↗First posted 20 Dec 2019cs.LG stat.MLcited by 92

Not on the map. Why?

Abstract (arXiv)

Attribution methods aim to explain a neural network's prediction by highlighting the most relevant image areas. A popular approach is to backpropagate (BP) a custom relevance score using modified rules, rather than the gradient. We analyze an extensive set of modified BP methods: Deep Taylor Decomposition, Layer-wise Relevance Propagation (LRP), Excitation BP, PatternAttribution, DeepLIFT, Deconv, RectGrad, and Guided BP. We find empirically that the explanations of all mentioned methods, except for DeepLIFT, are independent of the parameters of later layers. We provide theoretical insights for this surprising behavior and also analyze why DeepLIFT does not suffer from this limitation. Empirically, we measure how information of later layers is ignored by using our new metric, cosine similarity convergence (CSC). The paper provides a framework to assess the faithfulness of new and existing modified BP methods theoretically and empirically. For code see: https://github.com/berleon/when-explanations-lie

Key facts. Cited by 92 papers in the map (data through 14 Oct 2026). First posted on arXiv 20 Dec 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 21; 2022: 23; 2023: 16; 2024: 15; 2025: 12; 2026: 5.

92 papers.

Sort:

By area:

  1. 2022From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AIcited by 302
  2. 2022CRAFT: Concept Recursive Activation FacTorization for Explainabilitycited by 178
  3. 2022Quantus: An Explainable AI Toolkit for Responsible Evaluation of Neural Network Explanations and Beyondcited by 171
  4. 2023A Holistic Approach to Unifying Automatic Concept Extraction and Concept Importance Estimationcited by 109
  5. 2021Towards a Science of Human-AI Decision Making: A Survey of Empirical Studiescited by 86
  6. 2024A Primer on the Inner Workings of Transformer-based Language Modelscited by 85
  7. 2021XProtoNet: Diagnosis in Chest Radiography with Global and Local Explanationscited by 73
  8. 2022GlanceNets: Interpretabile, Leak-proof Concept-based Modelscited by 73
  9. 2025Archetypal SAE: Adaptive and Stable Dictionary Learning for Concept Extraction in Large Vision Modelscited by 73
  10. 2025Universal Sparse Autoencoders: Interpretable Cross-Model Concept Alignmentcited by 62
  11. 2022Leveraging Explanations in Interactive Machine Learning: An Overviewcited by 53
  12. 2021Do Input Gradients Highlight Discriminative Features?cited by 52
38 more
  1. 2022Concept-level Debugging of Part-Prototype Networkscited by 52
  2. 2022Logic-Based Explainability in Machine Learningcited by 48
  3. 2021This looks more like that: Enhancing Self-Explaining Models by Prototypical Relevance Propagationcited by 43
  4. 2022Rethinking Attention-Model Explainability through Faithfulness Violation Testcited by 43
  5. 2021Explaining a Series of Models by Propagating Shapley Valuescited by 37
  6. 2023The Meta-Evaluation Problem in Explainable AI: Identifying Reliable Estimators with MetaQuantuscited by 37
  7. 2021Neural Response Interpretation through the Lens of Critical Pathwayscited by 32
  8. 2023Finding the right XAI method -- A Guide for the Evaluation and Ranking of Explainable AI Methods in Climate Sciencecited by 27
  9. 2023Unlocking Feature Visualization for Deeper Networks with MAgnitude Constrained Optimizationcited by 27
  10. 2022Xplique: A Deep Learning Explainability Toolboxcited by 24
  11. 2022Shortcomings of Top-Down Randomization-Based Sanity Checks for Evaluations of Deep Neural Network Explanationscited by 22
  12. 2022Do Explanations Explain? Model Knows Bestcited by 20
  13. 2021Scrutinizing XAI using linear ground-truth data with suppressor variablescited by 18
  14. 2022On Computing Probabilistic Abductive Explanationscited by 18
  15. 2021Fine-Grained Neural Network Explanation by Identifying Input Features with Predictive Informationcited by 17
  16. 2023Right for the Wrong Reason: Can Interpretable ML Techniques Detect Spurious Correlations?cited by 16
  17. 2022A Fine-grained Interpretability Evaluation Benchmark for Neural NLPcited by 15
  18. 2025Why and How LLMs Hallucinate: Connecting the Dots with Subsequence Associationscited by 15
  19. 2023Explainability in AI Policies: A Critical Review of Communications, Reports, Regulations, and Standards in the EU, US, and UKcited by 13
  20. 2023Theoretical Behavior of XAI Methods in the Presence of Suppressor Variablescited by 12
  21. 2021Towards Rigorous Interpretations: a Formalisation of Feature Attributioncited by 11
  22. 2021Investigating sanity checks for saliency maps with image and text classificationcited by 11
  23. 2022Optimizing Explanations by Network Canonization and Hyperparameter Searchcited by 11
  24. 2023Pixel-Level Explanation of Multiple Instance Learning Models in Biomedical Single Cell Imagescited by 9
  25. 2024Sanity Checks Revisited: An Exploration to Repair the Model Parameter Randomisation Testcited by 9
  26. 2024A Fresh Look at Sanity Checks for Saliency Mapscited by 8
  27. 2022On the explainable properties of 1-Lipschitz Neural Networks: An Optimal Transport Perspectivecited by 6
  28. 2024Provably Better Explanations with Optimized Aggregation of Feature Attributionscited by 6
  29. 2021Evaluation of Saliency-based Explainability Methodcited by 5
  30. 2024Explainable AI needs formalizationcited by 5
  31. 2026Bi-Orthogonal Factor Decomposition for Vision Transformerscited by 5
  32. 2021Benchmarking Perturbation-based Saliency Maps for Explaining Atari Agentscited by 4
  33. 2021Toward a Unified Framework for Debugging Concept-based Modelscited by 4
  34. 2022Towards More Robust Interpretation via Local Gradient Alignmentcited by 4
  35. 2023A Survey of Explainable AI in Deep Visual Modeling: Methods and Metricscited by 4
  36. 2024Improving deep learning with prior knowledge and cognitive models: A survey on enhancing explainability, adversarial robustness and zero-shot learningcited by 4
  37. 2021Mutual Information Preserving Back-propagation: Learn to Invert for Faithful Attributioncited by 3
  38. 2021Do Deep Neural Networks Forget Facial Action Units? -- Exploring the Effects of Transfer Learning in Health Related Facial Expression Recognitioncited by 3

50 most cited of 92 All 92 (JSON)