# Explaining Data-Driven Decisions made by AI Systems: The Counterfactual Approach

arXiv: https://arxiv.org/abs/2001.07417 · HTML: https://mlsanity.com/paper/2001.07417/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Jan 2020.

## Built on by (most cited first, 26 of 26)

- [If Only We Had Better Counterfactual Explanations: Five Key Deficits to Rectify in the Evaluation of Counterfactual XAI Techniques](https://mlsanity.com/paper/2103.01035/) (2021)
- [NICE: An Algorithm for Nearest Instance Counterfactual Explanations](https://mlsanity.com/paper/2104.07411/) (2021)
- [Local Explanations via Necessity and Sufficiency: Unifying Theory and Practice](https://mlsanity.com/paper/2103.14651/) (2021)
- [To trust or not to trust an explanation: using LEAF to evaluate local linear XAI methods](https://mlsanity.com/paper/2106.00461/) (2021)
- [Tell Me a Story! Narrative-Driven XAI with Large Language Models](https://mlsanity.com/paper/2309.17057/) (2023)
- [A Framework and Benchmarking Study for Counterfactual Generating Methods on Tabular Data](https://mlsanity.com/paper/2107.04680/) (2021)
- [Counterfactual Explanations as Interventions in Latent Space](https://mlsanity.com/paper/2106.07754/) (2021)
- [Reinforced Path Reasoning for Counterfactual Explainable Recommendation](https://mlsanity.com/paper/2207.06674/) (2022)
- [Counterfactual Explanations via Latent Space Projection and Interpolation](https://mlsanity.com/paper/2112.00890/) (2021)
- [Sparse and Faithful Explanations Without Sparse Models](https://mlsanity.com/paper/2402.09702/) (2024)
- [COIN: Counterfactual Image Generation for VQA Interpretation](https://mlsanity.com/paper/2201.03342/) (2022)
- [Disagreement amongst counterfactual explanations: How transparency can be deceptive](https://mlsanity.com/paper/2304.12667/) (2023)
- [Counterfactual Explanations for Deep Learning-Based Traffic Forecasting](https://mlsanity.com/paper/2405.00456/) (2024)
- [CoCoG-2: Controllable generation of visual stimuli for understanding human concept representation](https://mlsanity.com/paper/2407.14949/) (2024)
- [Explainable AI for Psychological Profiling from Digital Footprints: A Case Study of Big Five Personality Predictions from Spending Data](https://mlsanity.com/paper/2111.06908/) (2021)
- [A General Search-based Framework for Generating Textual Counterfactual Explanations](https://mlsanity.com/paper/2211.00369/) (2022)
- [Unveiling the Potential of Counterfactuals Explanations in Employability](https://mlsanity.com/paper/2305.10069/) (2023)
- [Calculating and Visualizing Counterfactual Feature Importance Values](https://mlsanity.com/paper/2306.06506/) (2023)
- [Causality-based Counterfactual Explanation for Classification Models](https://mlsanity.com/paper/2105.00703/) (2021)
- [ComplAI: Theory of A Unified Framework for Multi-factor Assessment of Black-Box Supervised Machine Learning Models](https://mlsanity.com/paper/2212.14599/) (2022)
- [Counterfactual Explanation for Regression via Disentanglement in Latent Space](https://mlsanity.com/paper/2311.08228/) (2023)
- [Echoes of Biases: How Stigmatizing Language Affects AI Performance](https://mlsanity.com/paper/2305.10201/) (2023)
- [Counterfactual Explanation via Search in Gaussian Mixture Distributed Latent Space](https://mlsanity.com/paper/2307.13390/) (2023)
- [Deployment of a Robust and Explainable Mortality Prediction Model: The COVID-19 Pandemic and Beyond](https://mlsanity.com/paper/2311.17133/) (2023)
- [From What Ifs to Insights: Counterfactuals in Causal Inference vs. Explainable AI](https://mlsanity.com/paper/2505.13324/) (2025)
- [XAI-Arena: Can LLMs Assess the Quality of XAI Explanations?](https://mlsanity.com/paper/2609.09428/) (2026)

## Abstract (arXiv)

We examine counterfactual explanations for explaining the decisions made by model-based AI systems. The counterfactual approach we consider defines an explanation as a set of the system's data inputs that causally drives the decision (i.e., changing the inputs in the set changes the decision) and is irreducible (i.e., changing any subset of the inputs does not change the decision). We (1) demonstrate how this framework may be used to provide explanations for decisions made by general, data-driven AI systems that may incorporate features with arbitrary data types and multiple predictive models, and (2) propose a heuristic procedure to find the most useful explanations depending on the context. We then contrast counterfactual explanations with methods that explain model predictions by weighting features according to their importance (e.g., SHAP, LIME) and present two fundamental reasons why we should carefully consider whether importance-weight explanations are well-suited to explain system decisions. Specifically, we show that (i) features that have a large importance weight for a model prediction may not affect the corresponding decision, and (ii) importance weights are insufficient to communicate whether and how features influence decisions. We demonstrate this with several concise examples and three detailed case studies that compare the counterfactual approach with SHAP to illustrate various conditions under which counterfactual explanations explain data-driven decisions better than importance weights.