# Causal Explanations of Structural Causal Models

arXiv: https://arxiv.org/abs/2110.02395 · HTML: https://mlsanity.com/paper/2110.02395/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 47 references that are in the map.
- First posted on arXiv 5 Oct 2021.

## Builds on (most cited first, 15 of 15)

- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Counterfactual Fairness](https://mlsanity.com/paper/1703.06856/) (2017)
- [DAGs with NO TEARS: Continuous Optimization for Structure Learning](https://mlsanity.com/paper/1803.01422/) (2018)
- [This Looks Like That: Deep Learning for Interpretable Image Recognition](https://mlsanity.com/paper/1806.10574/) (2018)
- [Unmasking Clever Hans Predictors and Assessing What Machines Really Learn](https://mlsanity.com/paper/1902.10178/) (2019)
- [DAG-GNN: DAG Structure Learning with Graph Neural Networks](https://mlsanity.com/paper/1904.10098/) (2019)
- [Differentiable Causal Discovery from Interventional Data](https://mlsanity.com/paper/2007.01754/) (2020)
- [Learning Sparse Nonparametric DAGs](https://mlsanity.com/paper/1909.13189/) (2019)
- [Algorithmic recourse under imperfect causal knowledge: a probabilistic approach](https://mlsanity.com/paper/2006.06831/) (2020)
- [Learning Neural Causal Models from Unknown Interventions](https://mlsanity.com/paper/1910.01075/) (2019)
- [Restricting the Flow: Information Bottlenecks for Attribution](https://mlsanity.com/paper/2001.00396/) (2020)
- [CXPlain: Causal Explanations for Model Interpretation under Uncertainty](https://mlsanity.com/paper/1910.12336/) (2019)
- [The Causal-Neural Connection: Expressiveness, Learnability, and Inference](https://mlsanity.com/paper/2107.00793/) (2021)
- [Right for the Right Concept: Revising Neuro-Symbolic Concepts by Interacting with their Explanations](https://mlsanity.com/paper/2011.12854/) (2020)

## Built on by (most cited first, 4 of 4)

- [Artificial General Intelligence (AGI)-Native Wireless Systems: A Journey Beyond 6G](https://mlsanity.com/paper/2405.02336/) (2024)
- [Learning by Self-Explaining](https://mlsanity.com/paper/2309.08395/) (2023)
- [Causal Explanations Over Time: Articulated Reasoning for Interactive Environments](https://mlsanity.com/paper/2506.03915/) (2025)
- [Exploiting Constraint Reasoning to Build Graphical Explanations for Mixed-Integer Linear Programming](https://mlsanity.com/paper/2507.13007/) (2025)

## Abstract (arXiv)

In explanatory interactive learning (XIL) the user queries the learner, then the learner explains its answer to the user and finally the loop repeats. XIL is attractive for two reasons, (1) the learner becomes better and (2) the user's trust increases. For both reasons to hold, the learner's explanations must be useful to the user and the user must be allowed to ask useful questions. Ideally, both questions and explanations should be grounded in a causal model since they avoid spurious fallacies. Ultimately, we seem to seek a causal variant of XIL. The question part on the user's end we believe to be solved since the user's mental model can provide the causal model. But how would the learner provide causal explanations? In this work we show that existing explanation methods are not guaranteed to be causal even when provided with a Structural Causal Model (SCM). Specifically, we use the popular, proclaimed causal explanation method CXPlain to illustrate how the generated explanations leave open the question of truly causal explanations. Thus as a step towards causal XIL, we propose a solution to the lack of causal explanations. We solve this problem by deriving from first principles an explanation method that makes full use of a given SCM, which we refer to as SCE (E standing for explanation). Since SCEs make use of structural information, any causal graph learner can now provide human-readable explanations. We conduct several experiments including a user study with 22 participants to investigate the virtue of SCE as causal explanations of SCMs.