# Instance-wise Causal Feature Selection for Model Interpretation

arXiv: https://arxiv.org/abs/2104.12759 · HTML: https://mlsanity.com/paper/2104.12759/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 21 references that are in the map.
- First posted on arXiv 26 Apr 2021.

## Builds on (most cited first, 18 of 18)

- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [Sanity Checks for Saliency Maps](https://mlsanity.com/paper/1810.03292/) (2018)
- [The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables](https://mlsanity.com/paper/1611.00712/) (2016)
- [SmoothGrad: removing noise by adding noise](https://mlsanity.com/paper/1706.03825/) (2017)
- [Inverting Gradients -- How easy is it to break privacy in federated learning?](https://mlsanity.com/paper/2003.14053/) (2020)
- [Learning to Explain: An Information-Theoretic Perspective on Model Interpretation](https://mlsanity.com/paper/1802.07814/) (2018)
- [Explainable Machine Learning in Deployment](https://mlsanity.com/paper/1909.06342/) (2019)
- [Problems with Shapley-value-based explanations as feature importance measures](https://mlsanity.com/paper/2002.11097/) (2020)
- [Invariant Rationalization](https://mlsanity.com/paper/2003.09772/) (2020)
- [L-Shapley and C-Shapley: Efficient Model Interpretation for Structured Data](https://mlsanity.com/paper/1808.02610/) (2018)
- [CXPlain: Causal Explanations for Model Interpretation under Uncertainty](https://mlsanity.com/paper/1910.12336/) (2019)
- [Neural Network Attributions: A Causal Perspective](https://mlsanity.com/paper/1902.02302/) (2019)
- [Generative causal explanations of black-box classifiers](https://mlsanity.com/paper/2006.13913/) (2020)
- [Granger-causal Attentive Mixtures of Experts: Learning Important Features with Neural Networks](https://mlsanity.com/paper/1802.02195/) (2018)

## Built on by (most cited first, 8 of 8)

- [Deep Causal Learning: Representation, Discovery and Inference](https://mlsanity.com/paper/2211.03374/) (2022)
- [DIWIFT: Discovering Instance-wise Influential Features for Tabular Data](https://mlsanity.com/paper/2207.02773/) (2022)
- [Partial Information Decomposition for Data Interpretability and Feature Selection](https://mlsanity.com/paper/2405.19212/) (2024)
- [Causal Feature Selection for Responsible Machine Learning](https://mlsanity.com/paper/2402.02696/) (2024)
- [Causal Prototype-inspired Contrast Adaptation for Unsupervised Domain Adaptive Semantic Segmentation of High-resolution Remote Sensing Imagery](https://mlsanity.com/paper/2403.03704/) (2024)
- [Causality for Inherently Explainable Transformers: CAT-XPLAIN](https://mlsanity.com/paper/2206.14841/) (2022)
- [Unsupervised Dynamic Feature Selection for Robust Latent Spaces in Vision Tasks](https://mlsanity.com/paper/2510.01758/) (2025)
- [Explaining Black-Box Language Models: Learning to Optimize Linguistically-Structured Word Subsets](https://mlsanity.com/paper/2606.08497/) (2026)

## Abstract (arXiv)

We formulate a causal extension to the recently introduced paradigm of instance-wise feature selection to explain black-box visual classifiers. Our method selects a subset of input features that has the greatest causal effect on the models output. We quantify the causal influence of a subset of features by the Relative Entropy Distance measure. Under certain assumptions this is equivalent to the conditional mutual information between the selected subset and the output variable. The resulting causal selections are sparser and cover salient objects in the scene. We show the efficacy of our approach on multiple vision datasets by measuring the post-hoc accuracy and Average Causal Effect of selected features on the models output.