# Using Causal Analysis for Conceptual Deep Learning Explanation

arXiv: https://arxiv.org/abs/2107.06098 · HTML: https://mlsanity.com/paper/2107.06098/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 23 references that are in the map.
- First posted on arXiv 10 Jul 2021.

## Builds on (most cited first, 12 of 12)

- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison](https://mlsanity.com/paper/1901.07031/) (2019)
- [Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)](https://mlsanity.com/paper/1711.11279/) (2017)
- [Network Dissection: Quantifying Interpretability of Deep Visual Representations](https://mlsanity.com/paper/1704.05796/) (2017)
- [Direct and Indirect Effects](https://mlsanity.com/paper/1301.2300/) (2013)
- [Understanding the Role of Individual Units in a Deep Neural Network](https://mlsanity.com/paper/2009.05041/) (2020)
- [Explanation by Progressive Exaggeration](https://mlsanity.com/paper/1911.00483/) (2019)
- [Global and Local Interpretability for Cardiac MRI Classification](https://mlsanity.com/paper/1906.06188/) (2019)
- [Deep Learning for Screening COVID-19 using Chest X-Ray Images](https://mlsanity.com/paper/2004.10507/) (2020)

## Built on by (most cited first, 5 of 5)

- [Interpretable and Testable Vision Features via Sparse Autoencoders](https://mlsanity.com/paper/2502.06755/) (2025)
- [A Review of Causality for Learning Algorithms in Medical Image Analysis](https://mlsanity.com/paper/2206.05498/) (2022)
- [FastCAV: Efficient Computation of Concept Activation Vectors for Explaining Deep Neural Networks](https://mlsanity.com/paper/2505.17883/) (2025)
- [Transformer-based Personalized Attention Mechanism for Medical Images with Clinical Records](https://mlsanity.com/paper/2206.03003/) (2022)
- [Causal Transfer in Medical Image Analysis](https://mlsanity.com/paper/2603.24388/) (2026)

## Abstract (arXiv)

Model explainability is essential for the creation of trustworthy Machine Learning models in healthcare. An ideal explanation resembles the decision-making process of a domain expert and is expressed using concepts or terminology that is meaningful to the clinicians. To provide such an explanation, we first associate the hidden units of the classifier to clinically relevant concepts. We take advantage of radiology reports accompanying the chest X-ray images to define concepts. We discover sparse associations between concepts and hidden units using a linear sparse logistic regression. To ensure that the identified units truly influence the classifier's outcome, we adopt tools from Causal Inference literature and, more specifically, mediation analysis through counterfactual interventions. Finally, we construct a low-depth decision tree to translate all the discovered concepts into a straightforward decision rule, expressed to the radiologist. We evaluated our approach on a large chest x-ray dataset, where our model produces a global explanation consistent with clinical knowledge.