Neural Network Attributions: A Causal Perspective
Not on the map. Why?
Abstract (arXiv)
We propose a new attribution method for neural networks developed using first principles of causality (to the best of our knowledge, the first such). The neural network architecture is viewed as a Structural Causal Model, and a methodology to compute the causal effect of each feature on the output is presented. With reasonable assumptions on the causal structure of the input data, we propose algorithms to efficiently compute the causal effects, as well as scale the approach to data with large dimensionality. We also show how this method can be used for recurrent neural networks. We report experimental results on both simulated and real datasets showcasing the promise and usefulness of the proposed algorithm.
Key facts. Cited by 77 papers in the map (data through 7 Oct 2026). First posted on arXiv 6 Feb 2019.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 19; 2022: 14; 2023: 18; 2024: 7; 2025: 12; 2026: 7.
77 papers.
Sort:
By area:
- 2021Causal Abstractions of Neural Networkscited by 401
- 2023Causal Abstraction: A Theoretical Foundation for Mechanistic Interpretabilitycited by 194
- 2021Explainable artificial intelligence (XAI) in deep learning-based medical image analysiscited by 147
- 2021Generative Causal Explanations for Graph Neural Networkscited by 124
- 2021Inducing Causal Structure for Interpretable Neural Networkscited by 107
- 2021Socially Responsible AI Algorithms: Issues, Purposes, and Challengescited by 56
- 2022Leveraging Explanations in Interactive Machine Learning: An Overviewcited by 52
- 2022A Causal Lens for Controllable Text Generationcited by 45
- 2022Causality-based Neural Network Repaircited by 44
- 2023Rigorously Assessing Natural Language Explanations of Neuronscited by 44
- 2021Defining and Quantifying the Emergence of Sparse Concepts in DNNscited by 36
- 2022Reinforced Causal Explainer for Graph Neural Networkscited by 34
38 more
- 2023Towards Trustworthy and Aligned Machine Learning: A Data-centric Survey with Causality Perspectivescited by 30
- 2024Causal Concept Graph Models: Beyond Causal Opacity in Deep Learningcited by 28
- 2023Variational Information Pursuit for Interpretable Predictionscited by 25
- 2021On Causally Disentangled Representationscited by 22
- 2021Contrastive ACE: Domain Generalization Through Alignment of Causal Mechanismscited by 20
- 2022Adaptive Fairness Improvement Based on Causality Analysiscited by 17
- 2023Interpretation of Time-Series Deep Models: A Surveycited by 15
- 2025Why and How LLMs Hallucinate: Connecting the Dots with Subsequence Associationscited by 14
- 2024LLMScan: Causal Scan for LLM Misbehavior Detectioncited by 13
- 2026CausalArmor: Efficient Indirect Prompt Injection Guardrails via Causal Attributioncited by 13
- 2022On the Probability of Necessity and Sufficiency of Explaining Graph Neural Networks: A Lower Bound Optimization Approachcited by 11
- 2021Can We Faithfully Represent Masked States to Compute Shapley Values on a DNN?cited by 10
- 2021Matching Learned Causal Effects of Neural Networks with Domain Priorscited by 10
- 2025Interpretation Meets Safety: A Survey on Interpretation Methods and Tools for Improving LLM Safetycited by 10
- 2023Causality Analysis for Evaluating the Security of Large Language Modelscited by 9
- 2023Task-Driven Causal Feature Distillation: Towards Trustworthy Risk Predictioncited by 9
- 2021Instance-wise Causal Feature Selection for Model Interpretationcited by 8
- 2021Detecting Multi-Sensor Fusion Errors in Advanced Driver-Assistance Systemscited by 8
- 2023Causal Analysis for Robust Interpretability of Neural Networkscited by 8
- 2021Causal Learning for Socially Responsible AIcited by 7
- 2022Quality issues in Machine Learning Software Systemscited by 7
- 2021Improving Attribution Methods by Learning Submodular Functionscited by 5
- 2021Double Trouble: How to not explain a text classifier's decisions using counterfactuals synthesized by masked language models?cited by 5
- 2022Supporting Medical Relation Extraction via Causality-Pruned Semantic Dependency Forestcited by 5
- 2022On the Robustness of Explanations of Deep Neural Network Models: A Surveycited by 5
- 2023SUNY: A Visual Interpretation Framework for Convolutional Neural Networks from a Necessary and Sufficient Perspectivecited by 5
- 2024DiConStruct: Causal Concept-based Explanations through Black-Box Distillationcited by 5
- 2023Neuron Patching: Semantic-based Neuron-level Language Model Repair for Code Generationcited by 4
- 2023How Generalizable are Deepfake Image Detectors? An Empirical Studycited by 3
- 2023Emergence and Causality in Complex Systems: A Survey on Causal Emergence and Related Quantitative Studiescited by 3
- 2025Causal-Guided Detoxify Backdoor Attack of Open-Weight LoRA Modelscited by 3
- 2021Did I do that? Blame as a means to identify controlled effects in reinforcement learningcited by 2
- 2021DeepVisualInsight: Time-Travelling Visualization for Spatio-Temporal Causality of Deep Classification Trainingcited by 2
- 2023Towards Learning and Explaining Indirect Causal Effects in Neural Networkscited by 2
- 2023On the Multiple Roles of Ontologies in Explainable AIcited by 2
- 2024Causal Feature Selection for Responsible Machine Learningcited by 2
- 2025Attribution Explanations for Deep Neural Networks: A Theoretical Perspectivecited by 2
- 2026Causal Attribution via Activation Patchingcited by 2
50 most cited of 77 All 77 (JSON)
- 2026Decomposition of Evidence, Contradiction, and Fragility in Perturbation Responses13 Aug 2026
- 2026Discovering Efficient and Explainable Communication Topologies for LLM-based Multi-Agent Systems via Causal Inference13 Aug 2026
- 2026XtrAIn: Training-Guided Occlusion for Feature Attribution9 Jun 2026
- 2026A Closer Look at the Application of Causal Inference in Graph Representation Learning10 Apr 2026
- 2026Position: Explainable AI is Causality in Disguise30 Mar 2026
- 2026Causal Attribution via Activation Patching13 Mar 2026
- 2026CausalArmor: Efficient Indirect Prompt Injection Guardrails via Causal Attribution8 Feb 2026
- 2025A Model of Causal Explanation on Neural Networks for Tabular Data25 Dec 2025
- 2025Causal-Guided Detoxify Backdoor Attack of Open-Weight LoRA Models22 Dec 2025
- 2025CAuSE: Decoding Multimodal Classifiers using Faithful Natural Language Explanation7 Dec 2025
- 2025PragWorld: A Benchmark Evaluating LLMs' Local World Model under Minimal Linguistic Alterations and Conversational Dynamics17 Nov 2025
- 2025Explainable Graph Neural Networks: Understanding Brain Connectivity and Biomarkers in Dementia23 Sep 2025
38 more
- 2025Digging Into the Internal: Causality-Based Analysis of LLM Function Calling18 Sep 2025
- 2025Attribution Explanations for Deep Neural Networks: A Theoretical Perspective11 Aug 2025
- 2025Interpretation Meets Safety: A Survey on Interpretation Methods and Tools for Improving LLM Safety5 Jun 2025
- 2025Understanding the learned look-ahead behavior of chess neural networks26 May 2025
- 2025On Measuring Intrinsic Causal Attributions in Deep Neural Networks14 May 2025
- 2025Why and How LLMs Hallucinate: Connecting the Dots with Subsequence Associations17 Apr 2025
- 2025A Semantic-based Optimization Approach for Repairing LLMs: Case Study on Code Generation17 Mar 2025
- 2024LLMScan: Causal Scan for LLM Misbehavior Detection22 Oct 2024
- 2024Linking Model Intervention to Causal Interpretation in Model Explanation21 Oct 2024
- 2024Seeing Through VisualBERT: A Causal Adventure on Memetic Landscapes17 Oct 2024
- 2024Investigating and unmasking feature-level vulnerabilities of CNNs to adversarial perturbations31 May 2024
- 2024Causal Concept Graph Models: Beyond Causal Opacity in Deep Learning26 May 2024
- 2024Causal Feature Selection for Responsible Machine Learning5 Feb 2024
- 2024DiConStruct: Causal Concept-based Explanations through Black-Box Distillation16 Jan 2024
- 2023Emergence and Causality in Complex Systems: A Survey on Causal Emergence and Related Quantitative Studies28 Dec 2023
- 2023Task-Driven Causal Feature Distillation: Towards Trustworthy Risk Prediction20 Dec 2023
- 2023Rule-Extraction Methods From Feedforward Neural Networks: A Systematic Literature Review20 Dec 2023
- 2023Causality Analysis for Evaluating the Security of Large Language Models13 Dec 2023
- 2023Neuron Patching: Semantic-based Neuron-level Language Model Repair for Code Generation8 Dec 2023
- 2023On the Multiple Roles of Ontologies in Explainable AI8 Nov 2023
- 2023Rigorously Assessing Natural Language Explanations of Neurons19 Sep 2023
- 2023How Generalizable are Deepfake Image Detectors? An Empirical Study8 Aug 2023
- 2023Towards Trustworthy and Aligned Machine Learning: A Data-centric Survey with Causality Perspectives31 Jul 2023
- 2023Robust Ranking Explanations8 Jul 2023
- 2023Quality Issues in Machine Learning Software Systems26 Jun 2023
- 2023Interpretation of Time-Series Deep Models: A Survey23 May 2023
- 2023Causal Analysis for Robust Interpretability of Neural Networks15 May 2023
- 2023Towards Learning and Explaining Indirect Causal Effects in Neural Networks24 Mar 2023
- 2023Reasoning-Based Software Testing2 Mar 2023
- 2023SUNY: A Visual Interpretation Framework for Convolutional Neural Networks from a Necessary and Sufficient Perspective1 Mar 2023
- 2023Variational Information Pursuit for Interpretable Predictions6 Feb 2023
- 2023Causal Abstraction: A Theoretical Foundation for Mechanistic Interpretability11 Jan 2023
- 2022Provable Robust Saliency-based Explanations28 Dec 2022
- 2022On the Probability of Necessity and Sufficiency of Explaining Graph Neural Networks: A Lower Bound Optimization Approach14 Dec 2022
- 2022On the Robustness of Explanations of Deep Neural Network Models: A Survey9 Nov 2022
- 2022Causal Explanation for Reinforcement Learning: Quantifying State and Temporal Importance24 Oct 2022
- 2022Adaptive Fairness Improvement Based on Causality Analysis15 Sep 2022
- 2022Supporting Medical Relation Extraction via Causality-Pruned Semantic Dependency Forest29 Aug 2022
50 newest of 77 All 77 (JSON)
- 2026CausalArmor: Efficient Indirect Prompt Injection Guardrails via Causal Attributioncited by 13
- 2026Discovering Efficient and Explainable Communication Topologies for LLM-based Multi-Agent Systems via Causal Inferencecited by 1
- 2025Digging Into the Internal: Causality-Based Analysis of LLM Function Callingcited by 0
- 2026Discovering Efficient and Explainable Communication Topologies for LLM-based Multi-Agent Systems via Causal Inference13 Aug 2026
- 2026CausalArmor: Efficient Indirect Prompt Injection Guardrails via Causal Attribution8 Feb 2026
- 2025Digging Into the Internal: Causality-Based Analysis of LLM Function Calling18 Sep 2025
- 2021Causal Abstractions of Neural Networkscited by 401
- 2023Causal Abstraction: A Theoretical Foundation for Mechanistic Interpretabilitycited by 194
- 2021Explainable artificial intelligence (XAI) in deep learning-based medical image analysiscited by 147
- 2021Generative Causal Explanations for Graph Neural Networkscited by 124
- 2021Inducing Causal Structure for Interpretable Neural Networkscited by 107
- 2021Socially Responsible AI Algorithms: Issues, Purposes, and Challengescited by 56
- 2022Leveraging Explanations in Interactive Machine Learning: An Overviewcited by 52
- 2022A Causal Lens for Controllable Text Generationcited by 45
- 2023Rigorously Assessing Natural Language Explanations of Neuronscited by 44
- 2021Defining and Quantifying the Emergence of Sparse Concepts in DNNscited by 36
- 2022Reinforced Causal Explainer for Graph Neural Networkscited by 34
- 2024Causal Concept Graph Models: Beyond Causal Opacity in Deep Learningcited by 28
12 most cited of 69 All 69 (JSON)
- 2026Decomposition of Evidence, Contradiction, and Fragility in Perturbation Responses13 Aug 2026
- 2026XtrAIn: Training-Guided Occlusion for Feature Attribution9 Jun 2026
- 2026A Closer Look at the Application of Causal Inference in Graph Representation Learning10 Apr 2026
- 2026Position: Explainable AI is Causality in Disguise30 Mar 2026
- 2026Causal Attribution via Activation Patching13 Mar 2026
- 2025A Model of Causal Explanation on Neural Networks for Tabular Data25 Dec 2025
- 2025Causal-Guided Detoxify Backdoor Attack of Open-Weight LoRA Models22 Dec 2025
- 2025CAuSE: Decoding Multimodal Classifiers using Faithful Natural Language Explanation7 Dec 2025
- 2025Explainable Graph Neural Networks: Understanding Brain Connectivity and Biomarkers in Dementia23 Sep 2025
- 2025Attribution Explanations for Deep Neural Networks: A Theoretical Perspective11 Aug 2025
- 2025Understanding the learned look-ahead behavior of chess neural networks26 May 2025
- 2025On Measuring Intrinsic Causal Attributions in Deep Neural Networks14 May 2025
12 newest of 69 All 69 (JSON)