# Dear XAI Community, We Need to Talk! Fundamental Misconceptions in Current XAI Research

arXiv: https://arxiv.org/abs/2306.04292 · HTML: https://mlsanity.com/paper/2306.04292/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 48 references that are in the map.
- First posted on arXiv 7 Jun 2023.

## Builds on (most cited first, 22 of 22)

- [Shortcut Learning in Deep Neural Networks](https://mlsanity.com/paper/2004.07780/) (2020)
- [Towards A Rigorous Science of Interpretable Machine Learning](https://mlsanity.com/paper/1702.08608/) (2017)
- [Explanation in Artificial Intelligence: Insights from the Social Sciences](https://mlsanity.com/paper/1706.07269/) (2017)
- [Sanity Checks for Saliency Maps](https://mlsanity.com/paper/1810.03292/) (2018)
- [Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPR](https://mlsanity.com/paper/1711.00399/) (2017)
- [Adversarial Examples Are Not Bugs, They Are Features](https://mlsanity.com/paper/1905.02175/) (2019)
- [Network Dissection: Quantifying Interpretability of Deep Visual Representations](https://mlsanity.com/paper/1704.05796/) (2017)
- [Fooling LIME and SHAP: Adversarial Attacks on Post hoc Explanation Methods](https://mlsanity.com/paper/1911.02508/) (2019)
- [Interpretable Machine Learning: Fundamental Principles and 10 Grand Challenges](https://mlsanity.com/paper/2103.11251/) (2021)
- [The (Un)reliability of saliency methods](https://mlsanity.com/paper/1711.00867/) (2017)
- [Opportunities and Challenges in Explainable Artificial Intelligence (XAI): A Survey](https://mlsanity.com/paper/2006.11371/) (2020)
- [Compositional Explanations of Neurons](https://mlsanity.com/paper/2006.14032/) (2020)
- [True to the Model or True to the Data?](https://mlsanity.com/paper/2006.16234/) (2020)
- [Human-Centered Explainable AI (XAI): From Algorithms to User Experiences](https://mlsanity.com/paper/2110.10790/) (2021)
- ["How do I fool you?": Manipulating User Trust via Misleading Black Box Explanations](https://mlsanity.com/paper/1911.06473/) (2019)
- [Unrestricted Permutation forces Extrapolation: Variable Importance Requires at least One More Model, or There Is No Free Variable Importance](https://mlsanity.com/paper/1905.03151/) (2019)
- [Model-agnostic Feature Importance and Effects with Dependent Features -- A Conditional Subgroup Approach](https://mlsanity.com/paper/2006.04628/) (2020)
- [Relating the Partial Dependence Plot and Permutation Feature Importance to the Data Generating Process](https://mlsanity.com/paper/2109.01433/) (2021)
- [Scientific Inference With Interpretable Machine Learning: Analyzing Models to Learn About Real-World Phenomena](https://mlsanity.com/paper/2206.05487/) (2022)
- [Improvement-Focused Causal Recourse (ICR)](https://mlsanity.com/paper/2210.15709/) (2022)
- [Are there any 'object detectors' in the hidden layers of CNNs trained to identify objects or scenes?](https://mlsanity.com/paper/2007.01062/) (2020)
- [Rejecting Cognitivism: Computational Phenomenology for Deep Learning](https://mlsanity.com/paper/2302.09071/) (2023)

## Built on by (most cited first, 22 of 22)

- [Explainable Artificial Intelligence (XAI) 2.0: A Manifesto of Open Challenges and Interdisciplinary Research Directions](https://mlsanity.com/paper/2310.19775/) (2023)
- [Decoupling Pixel Flipping and Occlusion Strategy for Consistent XAI Benchmarks](https://mlsanity.com/paper/2401.06654/) (2024)
- [Towards Explainable Evaluation Metrics for Machine Translation](https://mlsanity.com/paper/2306.13041/) (2023)
- [Interpretable Machine Learning in Physics: A Review](https://mlsanity.com/paper/2503.23616/) (2025)
- [Towards Compositional Interpretability for XAI](https://mlsanity.com/paper/2406.17583/) (2024)
- [A Guide to Feature Importance Methods for Scientific Inference](https://mlsanity.com/paper/2404.12862/) (2024)
- [Rethinking Explainable Machine Learning as Applied Statistics](https://mlsanity.com/paper/2402.02870/) (2024)
- [Which LIME should I trust? Concepts, Challenges, and Solutions](https://mlsanity.com/paper/2503.24365/) (2025)
- [Is Trust Correlated With Explainability in AI? A Meta-Analysis](https://mlsanity.com/paper/2504.12529/) (2025)
- [Explainable AI needs formalization](https://mlsanity.com/paper/2409.14590/) (2024)
- [Information That Matters: Exploring Information Needs of People Affected by Algorithmic Decisions](https://mlsanity.com/paper/2401.13324/) (2024)
- [Causal hybrid modeling with double machine learning](https://mlsanity.com/paper/2402.13332/) (2024)
- [SIDEs: Separating Idealization from Deceptive Explanations in xAI](https://mlsanity.com/paper/2404.16534/) (2024)
- [A three-Level Framework for LLM-Enhanced eXplainable AI: From technical explanations to natural language](https://mlsanity.com/paper/2506.05887/) (2025)
- [BXRL: Behavior-Explainable Reinforcement Learning](https://mlsanity.com/paper/2603.23738/) (2026)
- [Statistics and explainability: a fruitful alliance](https://mlsanity.com/paper/2404.19301/) (2024)
- [How can we trust opaque systems? Criteria for robust explanations in XAI](https://mlsanity.com/paper/2508.12623/) (2025)
- [KTCF: Actionable Recourse in Knowledge Tracing via Counterfactual Explanations for Education](https://mlsanity.com/paper/2601.09156/) (2026)
- [Interpretability in Deep Time Series Models Demands Semantic Alignment](https://mlsanity.com/paper/2602.02239/) (2026)
- [cc-Shapley: Measuring Multivariate Feature Importance Needs Causal Context](https://mlsanity.com/paper/2602.20396/) (2026)
- [We Need Explanation Cards to Connect Explanation Algorithms to the Real World](https://mlsanity.com/paper/2606.16786/) (2026)
- [A Critical Analysis of Trustworthy AI Tools, Mark Frameworks, and the Implementation Chasms](https://mlsanity.com/paper/2607.15480/) (2026)

## Abstract (arXiv)

Despite progress in the field, significant parts of current XAI research are still not on solid conceptual, ethical, or methodological grounds. Unfortunately, these unfounded parts are not on the decline but continue to grow. Many explanation techniques are still proposed without clarifying their purpose. Instead, they are advertised with ever more fancy-looking heatmaps or only seemingly relevant benchmarks. Moreover, explanation techniques are motivated with questionable goals, such as building trust, or rely on strong assumptions about the 'concepts' that deep learning algorithms learn. In this paper, we highlight and discuss these and other misconceptions in current XAI research. We also suggest steps to make XAI a more substantive area of research.