mlsanityData to 14 Oct 2026

Dear XAI Community, We Need to Talk! Fundamental Misconceptions in Current XAI Research

arXiv 2306.04292 ↗First posted 7 Jun 2023cs.AI stat.MLcited by 22

Not on the map. Why?

Abstract (arXiv)

Despite progress in the field, significant parts of current XAI research are still not on solid conceptual, ethical, or methodological grounds. Unfortunately, these unfounded parts are not on the decline but continue to grow. Many explanation techniques are still proposed without clarifying their purpose. Instead, they are advertised with ever more fancy-looking heatmaps or only seemingly relevant benchmarks. Moreover, explanation techniques are motivated with questionable goals, such as building trust, or rely on strong assumptions about the 'concepts' that deep learning algorithms learn. In this paper, we highlight and discuss these and other misconceptions in current XAI research. We also suggest steps to make XAI a more substantive area of research.

Key facts. Cited by 22 papers in the map (data through 14 Oct 2026). Builds on 22 of its 48 references that are in the map. First posted on arXiv 7 Jun 2023.

Builds on

22 of 48 references.

References · 22

  1. 2020Shortcut Learning in Deep Neural Networkscited by 1,969
  2. 2017Towards A Rigorous Science of Interpretable Machine Learningcited by 1,416
  3. 2017Explanation in Artificial Intelligence: Insights from the Social Sciencescited by 1,206
  4. 2018Sanity Checks for Saliency Mapscited by 1,156
  5. 2017Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPRcited by 1,089
  6. 2019Adversarial Examples Are Not Bugs, They Are Featurescited by 953
  7. 2017Network Dissection: Quantifying Interpretability of Deep Visual Representationscited by 708
  8. 2019Fooling LIME and SHAP: Adversarial Attacks on Post hoc Explanation Methodscited by 443
  9. 2021Interpretable Machine Learning: Fundamental Principles and 10 Grand Challengescited by 367
  10. 2017The (Un)reliability of saliency methodscited by 333
  11. 2020Opportunities and Challenges in Explainable Artificial Intelligence (XAI): A Surveycited by 254
  12. 2020Compositional Explanations of Neuronscited by 172
10 more
  1. 2020True to the Model or True to the Data?cited by 126
  2. 2021Human-Centered Explainable AI (XAI): From Algorithms to User Experiencescited by 123
  3. 2019"How do I fool you?": Manipulating User Trust via Misleading Black Box Explanationscited by 111
  4. 2019Unrestricted Permutation forces Extrapolation: Variable Importance Requires at least One More Model, or There Is No Free Variable Importancecited by 87
  5. 2020Model-agnostic Feature Importance and Effects with Dependent Features -- A Conditional Subgroup Approachcited by 37
  6. 2021Relating the Partial Dependence Plot and Permutation Feature Importance to the Data Generating Processcited by 24
  7. 2022Scientific Inference With Interpretable Machine Learning: Analyzing Models to Learn About Real-World Phenomenacited by 21
  8. 2022Improvement-Focused Causal Recourse (ICR)cited by 20
  9. 2020Are there any 'object detectors' in the hidden layers of CNNs trained to identify objects or scenes?cited by 6
  10. 2023Rejecting Cognitivism: Computational Phenomenology for Deep Learningcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 9; 2025: 5; 2026: 6.

22 papers.

Sort:

By area:

  1. 2023Explainable Artificial Intelligence (XAI) 2.0: A Manifesto of Open Challenges and Interdisciplinary Research Directionscited by 125
  2. 2024Decoupling Pixel Flipping and Occlusion Strategy for Consistent XAI Benchmarkscited by 28
  3. 2023Towards Explainable Evaluation Metrics for Machine Translationcited by 15
  4. 2025Interpretable Machine Learning in Physics: A Reviewcited by 14
  5. 2024Towards Compositional Interpretability for XAIcited by 12
  6. 2024A Guide to Feature Importance Methods for Scientific Inferencecited by 11
  7. 2024Rethinking Explainable Machine Learning as Applied Statisticscited by 8
  8. 2025Which LIME should I trust? Concepts, Challenges, and Solutionscited by 8
  9. 2025Is Trust Correlated With Explainability in AI? A Meta-Analysiscited by 8
  10. 2024Explainable AI needs formalizationcited by 5
  11. 2024Information That Matters: Exploring Information Needs of People Affected by Algorithmic Decisionscited by 3
  12. 2024Causal hybrid modeling with double machine learningcited by 2
10 more
  1. 2024SIDEs: Separating Idealization from Deceptive Explanations in xAIcited by 2
  2. 2025A three-Level Framework for LLM-Enhanced eXplainable AI: From technical explanations to natural languagecited by 2
  3. 2026BXRL: Behavior-Explainable Reinforcement Learningcited by 1
  4. 2024Statistics and explainability: a fruitful alliancecited by 0
  5. 2025How can we trust opaque systems? Criteria for robust explanations in XAIcited by 0
  6. 2026KTCF: Actionable Recourse in Knowledge Tracing via Counterfactual Explanations for Educationcited by 0
  7. 2026Interpretability in Deep Time Series Models Demands Semantic Alignmentcited by 0
  8. 2026cc-Shapley: Measuring Multivariate Feature Importance Needs Causal Contextcited by 0
  9. 2026We Need Explanation Cards to Connect Explanation Algorithms to the Real Worldcited by 0
  10. 2026A Critical Analysis of Trustworthy AI Tools, Mark Frameworks, and the Implementation Chasmscited by 0