# Counterfactuals and Causability in Explainable Artificial Intelligence: Theory, Algorithms, and Applications

arXiv: https://arxiv.org/abs/2103.04244 · HTML: https://mlsanity.com/paper/2103.04244/

## Key facts

- Cited by 59 papers in the map (data through 14 Oct 2026).
- Builds on 69 of its 171 references that are in the map.
- First posted on arXiv 7 Mar 2021.

## Builds on (most cited first, 30 of 69)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI](https://mlsanity.com/paper/1910.10045/) (2019)
- [Learning Important Features Through Propagating Activation Differences](https://mlsanity.com/paper/1704.02685/) (2017)
- [Towards A Rigorous Science of Interpretable Machine Learning](https://mlsanity.com/paper/1702.08608/) (2017)
- [Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings](https://mlsanity.com/paper/1607.06520/) (2016)
- [Explanation in Artificial Intelligence: Insights from the Social Sciences](https://mlsanity.com/paper/1706.07269/) (2017)
- [Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPR](https://mlsanity.com/paper/1711.00399/) (2017)
- [A Survey Of Methods For Explaining Black Box Models](https://mlsanity.com/paper/1802.01933/) (2018)
- [Semantics derived automatically from language corpora contain human-like biases](https://mlsanity.com/paper/1608.07187/) (2016)
- [The Mythos of Model Interpretability](https://mlsanity.com/paper/1606.03490/) (2016)
- [Explaining Machine Learning Classifiers through Diverse Counterfactual Explanations](https://mlsanity.com/paper/1905.07697/) (2019)
- [Explaining Explanations: An Overview of Interpretability of Machine Learning](https://mlsanity.com/paper/1806.00069/) (2018)
- [Is Attention Interpretable?](https://mlsanity.com/paper/1906.03731/) (2019)
- [Confounding variables can degrade generalization performance of radiological deep learning models](https://mlsanity.com/paper/1807.00431/) (2018)
- [Unmasking Clever Hans Predictors and Assessing What Machines Really Learn](https://mlsanity.com/paper/1902.10178/) (2019)
- [European Union regulations on algorithmic decision-making and a "right to explanation"](https://mlsanity.com/paper/1606.08813/) (2016)
- [On the Robustness of Interpretability Methods](https://mlsanity.com/paper/1806.08049/) (2018)
- [Interpretable machine learning: definitions, methods, and applications](https://mlsanity.com/paper/1901.04592/) (2019)
- [Opportunities and Challenges in Explainable Artificial Intelligence (XAI): A Survey](https://mlsanity.com/paper/2006.11371/) (2020)
- [Word Embeddings Quantify 100 Years of Gender and Ethnic Stereotypes](https://mlsanity.com/paper/1711.08412/) (2017)
- [FACE: Feasible and Actionable Counterfactual Explanations](https://mlsanity.com/paper/1909.09369/) (2019)
- [Interpretable Counterfactual Explanations Guided by Prototypes](https://mlsanity.com/paper/1907.02584/) (2019)
- [The What-If Tool: Interactive Probing of Machine Learning Models](https://mlsanity.com/paper/1907.04135/) (2019)
- [Model-Agnostic Counterfactual Explanations for Consequential Decisions](https://mlsanity.com/paper/1905.11190/) (2019)
- [Avoiding Discrimination through Causal Reasoning](https://mlsanity.com/paper/1706.02744/) (2017)
- [Local Rule-Based Explanations of Black Box Decision Systems](https://mlsanity.com/paper/1805.10820/) (2018)
- [Multi-Objective Counterfactual Explanations](https://mlsanity.com/paper/2004.11165/) (2020)
- [Learning Model-Agnostic Counterfactual Explanations for Tabular Data](https://mlsanity.com/paper/1910.09398/) (2019)

## Built on by (most cited first, 30 of 59)

- [Black-Box Access is Insufficient for Rigorous AI Audits](https://mlsanity.com/paper/2401.14446/) (2024)
- [Explainable Artificial Intelligence (XAI) 2.0: A Manifesto of Open Challenges and Interdisciplinary Research Directions](https://mlsanity.com/paper/2310.19775/) (2023)
- [Causal Machine Learning for Healthcare and Precision Medicine](https://mlsanity.com/paper/2205.11402/) (2022)
- [Amortized Generation of Sequential Algorithmic Recourses for Black-box Models](https://mlsanity.com/paper/2106.03962/) (2021)
- [The role of causality in explainable artificial intelligence](https://mlsanity.com/paper/2309.09901/) (2023)
- [Features of Explainability: How users understand counterfactual and causal explanations for categorical and continuous features in XAI](https://mlsanity.com/paper/2204.10152/) (2022)
- [DiCE4EL: Interpreting Process Predictions using a Milestone-Aware Counterfactual Approach](https://mlsanity.com/paper/2107.08697/) (2021)
- [Counterfactual Generation with Identifiability Guarantees](https://mlsanity.com/paper/2402.15309/) (2024)
- [Robustness Implies Fairness in Causal Algorithmic Recourse](https://mlsanity.com/paper/2302.03465/) (2023)
- [Let's Go to the Alien Zoo: Introducing an Experimental Framework to Study Usability of Counterfactual Explanations for Machine Learning](https://mlsanity.com/paper/2205.03398/) (2022)
- [Beyond Known Reality: Exploiting Counterfactual Explanations for Medical Research](https://mlsanity.com/paper/2307.02131/) (2023)
- [Investigating the Duality of Interpretability and Explainability in Machine Learning](https://mlsanity.com/paper/2503.21356/) (2025)
- [Benchmarking Instance-Centric Counterfactual Algorithms for XAI: From White Box to Black Box](https://mlsanity.com/paper/2203.02399/) (2022)
- [Convex optimization for actionable & plausible counterfactual explanations](https://mlsanity.com/paper/2105.07630/) (2021)
- [Target specification bias, counterfactual prediction, and algorithmic fairness in healthcare](https://mlsanity.com/paper/2308.02081/) (2023)
- [What If We Had Used a Different App? Reliable Counterfactual KPI Analysis in Wireless Systems](https://mlsanity.com/paper/2410.00150/) (2024)
- [SAFE: Saliency-Aware Counterfactual Explanations for DNN-based Automated Driving Systems](https://mlsanity.com/paper/2307.15786/) (2023)
- [Causality from Bottom to Top: A Survey](https://mlsanity.com/paper/2403.11219/) (2024)
- [Pitfalls of Explainable ML: An Industry Perspective](https://mlsanity.com/paper/2106.07758/) (2021)
- [Liability regimes in the age of AI: a use-case driven analysis of the burden of proof](https://mlsanity.com/paper/2211.01817/) (2022)
- [Interpretable Knowledge Tracing via Response Influence-based Counterfactual Reasoning](https://mlsanity.com/paper/2312.10045/) (2023)
- [Generating Feasible and Plausible Counterfactual Explanations for Outcome Prediction of Business Processes](https://mlsanity.com/paper/2403.09232/) (2024)
- [Visual Analysis of Multi-outcome Causal Graphs](https://mlsanity.com/paper/2408.02679/) (2024)
- [Exploring the Trade-off between Plausibility, Change Intensity and Adversarial Power in Counterfactual Explanations using Multi-objective Optimization](https://mlsanity.com/paper/2205.10232/) (2022)
- [MDF-Net for abnormality detection by fusing X-rays with clinical data](https://mlsanity.com/paper/2302.13390/) (2023)
- [Counterfactual Explainer Framework for Deep Reinforcement Learning Models Using Policy Distillation](https://mlsanity.com/paper/2305.16532/) (2023)
- [Cultural Bias in Explainable AI Research: A Systematic Analysis](https://mlsanity.com/paper/2403.05579/) (2024)
- [Motif-Consistent Counterfactuals with Adversarial Refinement for Graph-Level Anomaly Detection](https://mlsanity.com/paper/2407.13251/) (2024)
- [Explainable AI over the Internet of Things (IoT): Overview, State-of-the-Art and Future Directions](https://mlsanity.com/paper/2211.01036/) (2022)
- [A Brief Review of Explainable Artificial Intelligence in Healthcare](https://mlsanity.com/paper/2304.01543/) (2023)

## Abstract (arXiv)

There has been a growing interest in model-agnostic methods that can make deep learning models more transparent and explainable to a user. Some researchers recently argued that for a machine to achieve a certain degree of human-level explainability, this machine needs to provide human causally understandable explanations, also known as causability. A specific class of algorithms that have the potential to provide causability are counterfactuals. This paper presents an in-depth systematic review of the diverse existing body of literature on counterfactuals and causability for explainable artificial intelligence. We performed an LDA topic modelling analysis under a PRISMA framework to find the most relevant literature articles. This analysis resulted in a novel taxonomy that considers the grounding theories of the surveyed algorithms, together with their underlying properties and applications in real-world data. This research suggests that current model-agnostic counterfactual algorithms for explainable AI are not grounded on a causal theoretical formalism and, consequently, cannot promote causability to a human decision-maker. Our findings suggest that the explanations derived from major algorithms in the literature provide spurious correlations rather than cause/effects relationships, leading to sub-optimal, erroneous or even biased explanations. This paper also advances the literature with new directions and challenges on promoting causability in model-agnostic approaches for explainable artificial intelligence.