mlsanityData to 7 Oct 2026

What Makes a Good Explanation?: A Harmonized View of Properties of Explanations

arXiv 2211.05667 ↗First posted 10 Nov 2022cs.LGcited by 21

Not on the map. Why?

Abstract (arXiv)

Interpretability provides a means for humans to verify aspects of machine learning (ML) models and empower human+ML teaming in situations where the task cannot be fully automated. Different contexts require explanations with different properties. For example, the kind of explanation required to determine if an early cardiac arrest warning system is ready to be integrated into a care setting is very different from the type of explanation required for a loan applicant to help determine the actions they might need to take to make their application successful. Unfortunately, there is a lack of standardization when it comes to properties of explanations: different papers may use the same term to mean different quantities, and different terms to mean the same quantity. This lack of a standardized terminology and categorization of the properties of ML explanations prevents us from both rigorously comparing interpretable machine learning methods and identifying what properties are needed in what contexts. In this work, we survey properties defined in interpretable machine learning papers, synthesize them based on what they actually measure, and describe the trade-offs between different formulations of these properties. In doing so, we enable more informed selection of task-appropriate formulations of explanation properties as well as standardization for future work in interpretable machine learning.

Key facts. Cited by 21 papers in the map (data through 7 Oct 2026). Builds on 69 of its 114 references that are in the map. First posted on arXiv 10 Nov 2022.

Builds on

69 of 114 references.

References · 69

  1. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  2. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  3. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  4. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  5. 2017Axiomatic Attribution for Deep Networkscited by 3,350
  6. 2015Learning Deep Features for Discriminative Localizationcited by 2,430
  7. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,348
  8. 2013Visualizing and Understanding Convolutional Networkscited by 2,278
  9. 2019Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AIcited by 1,543
  10. 2017Learning Important Features Through Propagating Activation Differencescited by 1,454
  11. 2018Sanity Checks for Saliency Mapscited by 1,149
  12. 2017SmoothGrad: removing noise by adding noisecited by 1,097
38 more
  1. 2014Striving for Simplicity: The All Convolutional Netcited by 983
  2. 2018A Survey Of Methods For Explaining Black Box Modelscited by 970
  3. 2018Robustness May Be at Odds with Accuracycited by 873
  4. 2018RISE: Randomized Input Sampling for Explanation of Black-box Modelscited by 825
  5. 2017Interpretable Explanations of Black Boxes by Meaningful Perturbationcited by 611
  6. 2019Score-CAM: Score-Weighted Visual Explanations for Convolutional Neural Networkscited by 575
  7. 2019ERASER: A Benchmark to Evaluate Rationalized NLP Modelscited by 523
  8. 2018Towards Robust Interpretability with Self-Explaining Neural Networkscited by 505
  9. 2017Interpretation of Neural Networks is Fragilecited by 443
  10. 2015Evaluating the visualization of what a Deep Neural Network has learnedcited by 416
  11. 2014Understanding Deep Image Representations by Inverting Themcited by 399
  12. 2020Explainability in Graph Neural Networks: A Taxonomic Surveycited by 383
  13. 2015Understanding Neural Networks Through Deep Visualizationcited by 332
  14. 2017The (Un)reliability of saliency methodscited by 331
  15. 2019On the (In)fidelity and Sensitivity for Explanationscited by 309
  16. 2022From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AIcited by 299
  17. 2020A Survey on Neural Network Interpretabilitycited by 291
  18. 2018Explanations based on the Missing: Towards Contrastive Explanations with Pertinent Negativescited by 261
  19. 2019On Completeness-aware Concept-Based Explanations in Deep Neural Networkscited by 239
  20. 2017Improving the Adversarial Robustness and Interpretability of Deep Neural Networks by Regularizing their Input Gradientscited by 236
  21. 2019Explanations can be manipulated and geometry is to blamecited by 215
  22. 2020Evaluating and Aggregating Feature-based Model Explanationscited by 185
  23. 2019One Explanation Does Not Fit All: A Toolkit and Taxonomy of AI Explainability Techniquescited by 164
  24. 2019Smooth Grad-CAM++: An Enhanced Inference Level Visualization Technique for Deep Convolutional Neural Network Modelscited by 118
  25. 2021Interpretable Deep Learning: Interpretation, Interpretability, Trustworthiness, and Beyondcited by 116
  26. 2019Fooling Neural Network Interpretations via Adversarial Model Manipulationcited by 112
  27. 2020The role of explainability in creating trustworthy artificial intelligence for health care: a comprehensive survey of the terminology, design choices, and evaluation strategiescited by 91
  28. 2021A Comprehensive Taxonomy for Explainable Artificial Intelligence: A Systematic Survey of Surveys on Methods and Conceptscited by 90
  29. 2021The Out-of-Distribution Problem in Explainability and Search Methods for Feature Importance Explanationscited by 86
  30. 2021Transparency of Deep Neural Networks for Medical Image Analysis: A Review of Interpretability Methodscited by 83
  31. 2019On the Connection Between Adversarial Robustness and Saliency Map Interpretabilitycited by 80
  32. 2018How do Humans Understand Explanations from Machine Learning Systems? An Evaluation of the Human-Interpretability of Explanationcited by 71
  33. 2020On quantitative aspects of model interpretabilitycited by 71
  34. 2018A Theoretical Explanation for Perplexing Behaviors of Backpropagation-based Visualizationscited by 59
  35. 2020Interpretability and Explainability: A Machine Learning Zoo Mini-tourcited by 55
  36. 2017Interpretable & Explorable Approximations of Black Box Modelscited by 54
  37. 2022Rethinking Stability for Attribution-based Explanationscited by 52
  38. 2022Improving the Fairness of Chest X-ray Classifierscited by 52

50 most cited of 69 All 69 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 5; 2024: 7; 2025: 5; 2026: 4.

21 papers.

Sort:

By area:

  1. 2024The Quest for the Right Mediator: Surveying Mechanistic Interpretability Through the Lens of Causal Mediation Analysiscited by 24
  2. 2024Helpful or Harmful Data? Fine-tuning-free Shapley Attribution for Explaining Language Model Predictionscited by 13
  3. 2023Why is plausibility surprisingly problematic as an XAI criterion?cited by 9
  4. 2023Argument Attribution Explanations in Quantitative Bipolar Argumentation Frameworks (Technical Report)cited by 7
  5. 2025Rethinking LLM Bias Probing Using Lessons from the Social Sciencescited by 6
  6. 2023Who's Thinking? A Push for Human-Centered Evaluation of LLMs using the XAI Playbookcited by 5
  7. 2024Investigating the Impact of Model Instability on Explanations and Uncertaintycited by 3
  8. 2024Explaining Arguments' Strength: Unveiling the Role of Attacks and Supports (Technical Report)cited by 3
  9. 2023Hidden Conflicts in Neural Networks and Their Implications for Explainabilitycited by 2
  10. 2024A Sim2Real Approach for Identifying Task-Relevant Properties in Interpretable Machine Learningcited by 2
  11. 2024Axiomatic Explainer Globalness via Optimal Transportcited by 2
  12. 2025Knowing What You Cannot Explain: Learning to Reject Low-Quality Explanationscited by 2
9 more
  1. 2025Attribution Explanations for Deep Neural Networks: A Theoretical Perspectivecited by 2
  2. 2025Informative Post-Hoc Explanations Only Exist for Simple Functionscited by 2
  3. 2023COMET: Neural Cost Model Explanation Frameworkcited by 1
  4. 2025Back to the Baseline: Examining Baseline Effects on Explainability Metricscited by 1
  5. 2026Assessing Model-Agnostic XAI Methods against EU AI Act Explainability Requirementscited by 1
  6. 2024Transparent Trade-offs between Properties of Explanationscited by 0
  7. 2026Textual Explanations and Their Evaluations for Reinforcement Learning Policycited by 0
  8. 2026GESD: Beyond Outcome-Oriented Fairnesscited by 0
  9. 2026Explainable AI for the EU Right to Explanation: A Systematic Review of the Law-XAI Translation Gapcited by 0