# The role of causality in explainable artificial intelligence

arXiv: https://arxiv.org/abs/2309.09901 · HTML: https://mlsanity.com/paper/2309.09901/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 136 references that are in the map.
- First posted on arXiv 18 Sep 2023.

## Builds on (most cited first, 30 of 38)

- [Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI](https://mlsanity.com/paper/1910.10045/) (2019)
- [Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPR](https://mlsanity.com/paper/1711.00399/) (2017)
- [A Survey Of Methods For Explaining Black Box Models](https://mlsanity.com/paper/1802.01933/) (2018)
- [The Mythos of Model Interpretability](https://mlsanity.com/paper/1606.03490/) (2016)
- [Explaining Machine Learning Classifiers through Diverse Counterfactual Explanations](https://mlsanity.com/paper/1905.07697/) (2019)
- [Algorithmic Recourse: from Counterfactual Explanations to Interventions](https://mlsanity.com/paper/2002.06278/) (2020)
- [Techniques for Interpretable Machine Learning](https://mlsanity.com/paper/1808.00033/) (2018)
- [Feature relevance quantification in explainable AI: A causal problem](https://mlsanity.com/paper/1910.13413/) (2019)
- [Explainable artificial intelligence (XAI) in deep learning-based medical image analysis](https://mlsanity.com/paper/2107.10912/) (2021)
- [Interpretable Machine Learning -- A Brief History, State-of-the-Art and Challenges](https://mlsanity.com/paper/2010.09337/) (2020)
- [Drug discovery with explainable artificial intelligence](https://mlsanity.com/paper/2007.00523/) (2020)
- [Causal Shapley Values: Exploiting Causal Knowledge to Explain Individual Predictions of Complex Models](https://mlsanity.com/paper/2011.01625/) (2020)
- [Causal Structure Learning](https://mlsanity.com/paper/1706.09141/) (2017)
- [Learning and Evaluating Graph Neural Network Explanations based on Counterfactual and Factual Reasoning](https://mlsanity.com/paper/2202.08816/) (2022)
- [General Pitfalls of Model-Agnostic Interpretation Methods for Machine Learning Models](https://mlsanity.com/paper/2007.04131/) (2020)
- [Theoretical Impediments to Machine Learning With Seven Sparks from the Causal Revolution](https://mlsanity.com/paper/1801.04016/) (2018)
- [Counterfactuals and Causability in Explainable Artificial Intelligence: Theory, Algorithms, and Applications](https://mlsanity.com/paper/2103.04244/) (2021)
- [Evaluating and Mitigating Bias in Image Classifiers: A Causal Perspective Using Counterfactuals](https://mlsanity.com/paper/2009.08270/) (2020)
- [Explaining Black-Box Algorithms Using Probabilistic Contrastive Counterfactuals](https://mlsanity.com/paper/2103.11972/) (2021)
- [Adversarial attacks and defenses in explainable artificial intelligence: A survey](https://mlsanity.com/paper/2306.06123/) (2023)
- [Local Explanations via Necessity and Sufficiency: Unifying Theory and Practice](https://mlsanity.com/paper/2103.14651/) (2021)
- [Explainable artificial intelligence model to predict acute critical illness from electronic health records](https://mlsanity.com/paper/1912.01266/) (2019)
- [How Well do Feature Visualizations Support Causal Understanding of CNN Activations?](https://mlsanity.com/paper/2106.12447/) (2021)
- [Survey of explainable machine learning with visual and granular methods beyond quasi-explanations](https://mlsanity.com/paper/2009.10221/) (2020)
- [Causal Deep Learning](https://mlsanity.com/paper/2303.02186/) (2023)
- [Rational Shapley Values](https://mlsanity.com/paper/2106.10191/) (2021)
- [Agree to Disagree: When Deep Learning Models With Identical Architectures Produce Distinct Explanations](https://mlsanity.com/paper/2105.06791/) (2021)
- [A multi-component framework for the analysis and design of explainable artificial intelligence](https://mlsanity.com/paper/2005.01908/) (2020)
- [Explanations for Occluded Images](https://mlsanity.com/paper/2103.03622/) (2021)
- [Outcome-Explorer: A Causality Guided Interactive Visual Interface for Interpretable Algorithmic Decision Making](https://mlsanity.com/paper/2101.00633/) (2021)

## Built on by (most cited first, 17 of 17)

- [Towards Compositional Interpretability for XAI](https://mlsanity.com/paper/2406.17583/) (2024)
- [Explain the Black Box for the Sake of Science: the Scientific Method in the Era of Generative Artificial Intelligence](https://mlsanity.com/paper/2406.10557/) (2024)
- [Survey on AI Ethics: A Socio-technical Perspective](https://mlsanity.com/paper/2311.17228/) (2023)
- [Explainable AI: Learning from the Learners](https://mlsanity.com/paper/2601.05525/) (2026)
- [Attribution Explanations for Deep Neural Networks: A Theoretical Perspective](https://mlsanity.com/paper/2508.07636/) (2025)
- [GraphRAG-Causal: A novel graph-augmented framework for causal reasoning and annotation in news](https://mlsanity.com/paper/2506.11600/) (2025)
- [Chain-of-Image Generation: Toward Monitorable and Controllable Image Generation](https://mlsanity.com/paper/2512.08645/) (2025)
- [What is Missing? Explaining Neurons Activated by Absent Concepts](https://mlsanity.com/paper/2603.09787/) (2026)
- [Predicting Trust Dynamics with Dynamic SEM in Human-AI Cooperation](https://mlsanity.com/paper/2407.01752/) (2024)
- [Interpretable Neural System Dynamics: Combining Deep Learning with System Dynamics Modeling to Support Critical Applications](https://mlsanity.com/paper/2505.14428/) (2025)
- [OpenAg: Democratizing Agricultural Intelligence](https://mlsanity.com/paper/2506.04571/) (2025)
- [Towards explainable decision support using hybrid neural models for logistic terminal automation](https://mlsanity.com/paper/2509.07577/) (2025)
- [Integrating a Causal Foundation Model into a Prescriptive Maintenance Framework for Optimising Production-Line OEE](https://mlsanity.com/paper/2512.00969/) (2025)
- [cc-Shapley: Measuring Multivariate Feature Importance Needs Causal Context](https://mlsanity.com/paper/2602.20396/) (2026)
- [Position: Explainable AI is Causality in Disguise](https://mlsanity.com/paper/2603.28597/) (2026)
- [Hierarchical Causal Abduction: A Foundation Framework for Explainable Model Predictive Control](https://mlsanity.com/paper/2605.10624/) (2026)
- [A Step Towards Inherently Interpretable Causal Machine Learning Models For Decision Support](https://mlsanity.com/paper/2606.24348/) (2026)

## Abstract (arXiv)

Causality and eXplainable Artificial Intelligence (XAI) have developed as separate fields in computer science, even though the underlying concepts of causation and explanation share common ancient roots. This is further enforced by the lack of review works jointly covering these two fields. In this paper, we investigate the literature to try to understand how and to what extent causality and XAI are intertwined. More precisely, we seek to uncover what kinds of relationships exist between the two concepts and how one can benefit from them, for instance, in building trust in AI systems. As a result, three main perspectives are identified. In the first one, the lack of causality is seen as one of the major limitations of current AI and XAI approaches, and the "optimal" form of explanations is investigated. The second is a pragmatic perspective and considers XAI as a tool to foster scientific exploration for causal inquiry, via the identification of pursue-worthy experimental manipulations. Finally, the third perspective supports the idea that causality is propaedeutic to XAI in three possible manners: exploiting concepts borrowed from causality to support or improve XAI, utilizing counterfactuals for explainability, and considering accessing a causal model as explaining itself. To complement our analysis, we also provide relevant software solutions used to automate causal tasks. We believe our work provides a unified view of the two fields of causality and XAI by highlighting potential domain bridges and uncovering possible limitations.