mlsanityData to 14 Oct 2026

On the Relationship Between Interpretability and Explainability in Machine Learning

arXiv 2311.11491 ↗First posted 20 Nov 2023cs.LG cs.AIcited by 3

Not on the map. Why?

Abstract (arXiv)

Interpretability and explainability have gained more and more attention in the field of machine learning as they are crucial when it comes to high-stakes decisions and troubleshooting. Since both provide information about predictors and their decision process, they are often seen as two independent means for one single end. This view has led to a dichotomous literature: explainability techniques designed for complex black-box models, or interpretable approaches ignoring the many explainability tools. In this position paper, we challenge the common idea that interpretability and explainability are substitutes for one another by listing their principal shortcomings and discussing how both of them mitigate the drawbacks of the other. In doing so, we call for a new perspective on interpretability and explainability, and works targeting both topics simultaneously, leveraging each of their respective assets.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 64 of its 123 references that are in the map. First posted on arXiv 20 Nov 2023.

Builds on

64 of 123 references.

References · 64

  1. 2017A Unified Approach to Interpreting Model Predictionscited by 6,555
  2. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,070
  3. 2014Going Deeper with Convolutionscited by 4,241
  4. 2019Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AIcited by 1,552
  5. 2017Learning Important Features Through Propagating Activation Differencescited by 1,458
  6. 2017Towards A Rigorous Science of Interpretable Machine Learningcited by 1,416
  7. 2017Explanation in Artificial Intelligence: Insights from the Social Sciencescited by 1,206
  8. 2018Sanity Checks for Saliency Mapscited by 1,156
  9. 2017Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPRcited by 1,089
  10. 2018A Survey Of Methods For Explaining Black Box Modelscited by 973
  11. 2019Attention is not Explanationcited by 929
  12. 2015The Limitations of Deep Learning in Adversarial Settingscited by 790
38 more
  1. 2016The Mythos of Model Interpretabilitycited by 721
  2. 2015Distillation as a Defense to Adversarial Perturbations against Deep Neural Networkscited by 678
  3. 2016Distribution-Free Predictive Inference For Regressioncited by 668
  4. 2017Interpretable Explanations of Black Boxes by Meaningful Perturbationcited by 612
  5. 2018Explaining Explanations: An Overview of Interpretability of Machine Learningcited by 508
  6. 2017Interpretation of Neural Networks is Fragilecited by 444
  7. 2019Fooling LIME and SHAP: Adversarial Attacks on Post hoc Explanation Methodscited by 443
  8. 2017Residual Attention Network for Image Classificationcited by 386
  9. 2018All Models are Wrong, but Many are Useful: Learning a Variable's Importance by Studying an Entire Class of Prediction Models Simultaneouslycited by 373
  10. 2021Interpretable Machine Learning: Fundamental Principles and 10 Grand Challengescited by 367
  11. 2016European Union regulations on algorithmic decision-making and a "right to explanation"cited by 344
  12. 2020A Survey on Neural Network Interpretabilitycited by 291
  13. 2018Consistent Individualized Feature Attribution for Tree Ensemblescited by 285
  14. 2020Opportunities and Challenges in Explainable Artificial Intelligence (XAI): A Surveycited by 254
  15. 2015Explaining NonLinear Classification Decisions with Deep Taylor Decompositioncited by 250
  16. 2017Real Time Image Saliency for Black Box Classifierscited by 246
  17. 2016Visualizing the Effects of Predictor Variables in Black Box Supervised Learning Modelscited by 213
  18. 2020Multi-Objective Counterfactual Explanationscited by 157
  19. 2020Explainable Deep Learning: A Field Guide for the Uninitiatedcited by 127
  20. 2021Explainable AI (XAI): A Systematic Meta-Survey of Current Challenges and Future Opportunitiescited by 125
  21. 2019"How do I fool you?": Manipulating User Trust via Misleading Black Box Explanationscited by 111
  22. 2019The Dangers of Post-hoc Interpretability: Unjustified Counterfactual Explanationscited by 97
  23. 2020The role of explainability in creating trustworthy artificial intelligence for health care: a comprehensive survey of the terminology, design choices, and evaluation strategiescited by 92
  24. 2019Pruning by Explaining: A Novel Criterion for Deep Neural Network Pruningcited by 89
  25. 2020On Interpretability of Artificial Neural Networks: A Surveycited by 88
  26. 2019Unrestricted Permutation forces Extrapolation: Variable Importance Requires at least One More Model, or There Is No Free Variable Importancecited by 87
  27. 2017Learning how to explain neural networks: PatternNet and PatternAttributioncited by 84
  28. 2020General Pitfalls of Model-Agnostic Interpretation Methods for Machine Learning Modelscited by 75
  29. 2019Exploratory Not Explanatory: Counterfactual Analysis of Saliency Maps for Deep Reinforcement Learningcited by 60
  30. 2017SPINE: SParse Interpretable Neural Embeddingscited by 55
  31. 2018Visualizing the Feature Importance for Black Box Modelscited by 55
  32. 2017Interpretable & Explorable Approximations of Black Box Modelscited by 54
  33. 2020Towards falsifiable interpretability researchcited by 53
  34. 2022Post-Hoc Explanations Fail to Achieve their Purpose in Adversarial Contextscited by 53
  35. 2018A Simple and Effective Model-Based Variable Importance Measurecited by 51
  36. 2020Model-agnostic Feature Importance and Effects with Dependent Features -- A Conditional Subgroup Approachcited by 37
  37. 2019Enhancing Explainability of Neural Networks through Architecture Constraintscited by 29
  38. 2015Exploring the Space of Adversarial Imagescited by 28

50 most cited of 64 All 64 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 2.

3 papers.

  1. 2025A Survey on Interpretability in Visual Recognitioncited by 2
  2. 2026Procedural Fairness via Group Counterfactual Explanationcited by 1
  3. 2026Uncertainty-Aware Transformers: Conformal Prediction for Language Modelscited by 0