Position: Explainable AI is Causality in Disguise
Not on the map. Why?
Abstract (arXiv)
The demand for Explainable AI (XAI) has triggered an explosion of methods, producing a landscape so fragmented that we now rely on surveys of surveys. Yet, fundamental challenges persist: conflicting metrics, failed sanity checks, and unresolved debates over robustness and fairness. The only consensus on how to achieve explainability is a lack of one. This has led many to point to the absence of a ground truth for defining ``the'' correct explanation as the main culprit. This position paper posits that the persistent discord in XAI arises not from an absent ground truth but from a ground truth that exists, albeit as an elusive and challenging target: the causal model that governs the relevant system. By reframing XAI queries about data, models, or decisions as causal inquiries, we prove the necessity and sufficiency of causal models for XAI. We contend that without this causal grounding, XAI remains unmoored. Ultimately, we encourage the community to converge around advanced concept and causal discovery to escape this entrenched uncertainty.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 66 of its 117 references that are in the map. First posted on arXiv 30 Mar 2026.
Builds on
66 of 117 references.
References · 66
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
- 2017A Unified Approach to Interpreting Model Predictionscited by 6,555
- 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,070
- 2017Axiomatic Attribution for Deep Networkscited by 3,384
- 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,898
- 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,371
- 2015Variational Inference with Normalizing Flowscited by 2,170
- 2019Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AIcited by 1,552
- 2017Towards A Rigorous Science of Interpretable Machine Learningcited by 1,416
- 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,319
- 2017Explanation in Artificial Intelligence: Insights from the Social Sciencescited by 1,206
- 2018Sanity Checks for Saliency Mapscited by 1,156
38 more
- 2017Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPRcited by 1,089
- 2020Concept Bottleneck Modelscited by 1,032
- 2018A Survey Of Methods For Explaining Black Box Modelscited by 973
- 2018Explaining Explanations: An Overview of Interpretability of Machine Learningcited by 508
- 2017Interpretation of Neural Networks is Fragilecited by 444
- 2019Fooling LIME and SHAP: Adversarial Attacks on Post hoc Explanation Methodscited by 443
- 2019A Survey on Explainable Artificial Intelligence (XAI): Towards Medical XAIcited by 404
- 2017Methods for Interpreting and Understanding Deep Neural Networkscited by 345
- 2015Understanding Neural Networks Through Deep Visualizationcited by 334
- 2017The (Un)reliability of saliency methodscited by 333
- 2019On the (In)fidelity and Sensitivity for Explanationscited by 310
- 2019Interpretable machine learning: definitions, methods, and applicationscited by 305
- 2018Metrics for Explainable AI: Challenges and Prospectscited by 260
- 2020Algorithmic Recourse: from Counterfactual Explanations to Interventionscited by 250
- 2018Techniques for Interpretable Machine Learningcited by 238
- 2019Explanations can be manipulated and geometry is to blamecited by 216
- 2016Visualizing the Effects of Predictor Variables in Black Box Supervised Learning Modelscited by 213
- 2021D'ya like DAGs? A Survey on Structure Learning and Causal Discoverycited by 211
- 2023Causal Abstraction: A Theoretical Foundation for Mechanistic Interpretabilitycited by 205
- 2019Feature relevance quantification in explainable AI: A causal problemcited by 167
- 2019A Meta-Transfer Objective for Learning to Disentangle Causal Mechanismscited by 158
- 2020Algorithmic recourse under imperfect causal knowledge: a probabilistic approachcited by 140
- 2018Explaining Explanations in AIcited by 137
- 2019Explaining Classifiers with Causal Concept Effect (CaCE)cited by 135
- 2019Sanity Checks for Saliency Metricscited by 119
- 2017Causal Consistency of Structural Equation Modelscited by 118
- 2019Fooling Neural Network Interpretations via Adversarial Model Manipulationcited by 112
- 2018Abstracting Causal Modelscited by 107
- 2020Explaining Explanations: Axiomatic Feature Interactions for Deep Networkscited by 103
- 2019The Dangers of Post-hoc Interpretability: Unjustified Counterfactual Explanationscited by 97
- 2019Neural Network Attributions: A Causal Perspectivecited by 77
- 2015The Bayesian Case Model: A Generative Approach for Case-Based Reasoning and Prototype Classificationcited by 76
- 2020Model Interpretability through the Lens of Computational Complexitycited by 75
- 2020On the Fairness of Causal Algorithmic Recoursecited by 70
- 2023Not All Neuro-Symbolic Concepts Are Created Equal: Analysis and Mitigation of Reasoning Shortcutscited by 63
- 2021Counterfactuals and Causability in Explainable Artificial Intelligence: Theory, Algorithms, and Applicationscited by 59
- 2020Interpretability and Explainability: A Machine Learning Zoo Mini-tourcited by 55
- 2022Leveraging Explanations in Interactive Machine Learning: An Overviewcited by 53
50 most cited of 66 All 66 (JSON)
Built on by
Not cited yet.