Linking Model Intervention to Causal Interpretation in Model Explanation
Not on the map. Why?
Abstract (arXiv)
Intervention intuition is often used in model explanation where the intervention effect of a feature on the outcome is quantified by the difference of a model prediction when the feature value is changed from the current value to the baseline value. Such a model intervention effect of a feature is inherently association. In this paper, we will study the conditions when an intuitive model intervention effect has a causal interpretation, i.e., when it indicates whether a feature is a direct cause of the outcome. This work links the model intervention effect to the causal interpretation of a model. Such an interpretation capability is important since it indicates whether a machine learning model is trustworthy to domain experts. The conditions also reveal the limitations of using a model intervention effect for causal interpretation in an environment with unobserved features. Experiments on semi-synthetic datasets have been conducted to validate theorems and show the potential for using the model intervention effect for model interpretation.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 27 of its 52 references that are in the map. First posted on arXiv 21 Oct 2024.
Builds on
27 of 52 references.
References · 27
- 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,733
- 2017A Unified Approach to Interpreting Model Predictionscited by 6,555
- 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,070
- 2019Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AIcited by 1,552
- 2017Learning Important Features Through Propagating Activation Differencescited by 1,458
- 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,319
- 2017Explanation in Artificial Intelligence: Insights from the Social Sciencescited by 1,206
- 2019The many Shapley values for model explanationcited by 344
- 2019Interpretable machine learning: definitions, methods, and applicationscited by 305
- 2018Techniques for Interpretable Machine Learningcited by 238
- 2016Understanding Neural Networks through Representation Erasurecited by 224
- 2019Feature relevance quantification in explainable AI: A causal problemcited by 167
15 more
- 2017Visualizing Deep Neural Network Decisions: Prediction Difference Analysiscited by 166
- 2019Explaining Classifiers with Causal Concept Effect (CaCE)cited by 135
- 2020CausaLM: Causal Model Explanation Through Counterfactual Language Modelscited by 117
- 2020Causal Shapley Values: Exploiting Causal Knowledge to Explain Individual Predictions of Complex Modelscited by 107
- 2019CXPlain: Causal Explanations for Model Interpretation under Uncertaintycited by 98
- 2020Causal Interpretability for Machine Learning -- Problems, Methods and Evaluationcited by 82
- 2019Neural Network Attributions: A Causal Perspectivecited by 77
- 2022CEBaB: Estimating the Causal Effects of Real-World Concepts on NLP Model Behaviorcited by 55
- 2020Generative causal explanations of black-box classifierscited by 41
- 2019Global Explanations of Neural Networks: Mapping the Landscape of Predictionscited by 32
- 2014Bayesian Network Constraint-Based Structure Learning Algorithms: Parallel and Optimised Implementations in the bnlearn R Packagecited by 19
- 2018Global Model Interpretation via Recursive Partitioningcited by 17
- 2018Explaining Deep Learning Models - A Bayesian Non-parametric Approachcited by 16
- 2018Granger-causal Attentive Mixtures of Experts: Learning Important Features with Neural Networkscited by 13
- 2017MAGIX: Model Agnostic Globally Interpretable Explanationscited by 3
Built on by
Citing papers per year: 2026: 1.
1 paper.