# OrphicX: A Causality-Inspired Latent Variable Model for Interpreting Graph Neural Networks

arXiv: https://arxiv.org/abs/2203.15209 · HTML: https://mlsanity.com/paper/2203.15209/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 35 references that are in the map.
- First posted on arXiv 29 Mar 2022.

## Builds on (most cited first, 21 of 21)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [How Powerful are Graph Neural Networks?](https://mlsanity.com/paper/1810.00826/) (2018)
- [Variational Graph Auto-Encoders](https://mlsanity.com/paper/1611.07308/) (2016)
- [GNNExplainer: Generating Explanations for Graph Neural Networks](https://mlsanity.com/paper/1903.03894/) (2019)
- [Geometric deep learning on graphs and manifolds using mixture model CNNs](https://mlsanity.com/paper/1611.08402/) (2016)
- [Parameterized Explainer for Graph Neural Network](https://mlsanity.com/paper/2011.04573/) (2020)
- [On Explainability of Graph Neural Networks via Subgraph Explorations](https://mlsanity.com/paper/2102.05152/) (2021)
- [XGNN: Towards Model-Level Explanations of Graph Neural Networks](https://mlsanity.com/paper/2006.02587/) (2020)
- [PGM-Explainer: Probabilistic Graphical Model Explanations for Graph Neural Networks](https://mlsanity.com/paper/2010.05788/) (2020)
- [Interpreting Graph Neural Networks for NLP With Differentiable Edge Masking](https://mlsanity.com/paper/2010.00577/) (2020)
- [Generative Causal Explanations for Graph Neural Networks](https://mlsanity.com/paper/2104.06643/) (2021)
- [Graph Information Bottleneck for Subgraph Recognition](https://mlsanity.com/paper/2010.05563/) (2020)
- [Generative Interventions for Causal Learning](https://mlsanity.com/paper/2012.12265/) (2020)
- [Generative causal explanations of black-box classifiers](https://mlsanity.com/paper/2006.13913/) (2020)
- [Causal Discovery from Incomplete Data: A Deep Learning Approach](https://mlsanity.com/paper/2001.05343/) (2020)
- [Temporal Dynamic Graph LSTM for Action-driven Video Object Detection](https://mlsanity.com/paper/1708.00666/) (2017)
- [Correcting Exposure Bias for Link Recommendation](https://mlsanity.com/paper/2106.07041/) (2021)
- [Deep Graph Random Process for Relational-Thinking-Based Speech Recognition](https://mlsanity.com/paper/2007.02126/) (2020)

## Built on by (most cited first, 30 of 33)

- [Trustworthy Graph Neural Networks: Aspects, Methods and Trends](https://mlsanity.com/paper/2205.07424/) (2022)
- [Global Explainability of GNNs via Logic Combination of Learned Concepts](https://mlsanity.com/paper/2210.07147/) (2022)
- [Explaining the Explainers in Graph Neural Networks: a Comparative Study](https://mlsanity.com/paper/2210.15304/) (2022)
- [GNNInterpreter: A Probabilistic Generative Model-Level Explanation for Graph Neural Networks](https://mlsanity.com/paper/2209.07924/) (2022)
- [Energy-Based Concept Bottleneck Models: Unifying Prediction, Concept Intervention, and Probabilistic Interpretations](https://mlsanity.com/paper/2401.14142/) (2024)
- [A Survey on Explainability of Graph Neural Networks](https://mlsanity.com/paper/2306.01958/) (2023)
- [PaGE-Link: Path-based Graph Neural Network Explanation for Heterogeneous Link Prediction](https://mlsanity.com/paper/2302.12465/) (2023)
- [Self-Interpretable Time Series Prediction with Counterfactual Explanations](https://mlsanity.com/paper/2306.06024/) (2023)
- [Causal Transportability for Visual Recognition](https://mlsanity.com/paper/2204.12363/) (2022)
- [Distill n' Explain: explaining graph neural networks using simple surrogates](https://mlsanity.com/paper/2303.10139/) (2023)
- [CI-GNN: A Granger Causality-Inspired Graph Neural Network for Interpretable Brain Network-Based Psychiatric Diagnosis](https://mlsanity.com/paper/2301.01642/) (2023)
- [Dynamic Graph Representation Learning with Neural Networks: A Survey](https://mlsanity.com/paper/2304.05729/) (2023)
- [How Interpretable Are Interpretable Graph Neural Networks?](https://mlsanity.com/paper/2406.07955/) (2024)
- [Graph Neural Network Causal Explanation via Neural Causal Models](https://mlsanity.com/paper/2407.09378/) (2024)
- [Exploring Causal Learning through Graph Neural Networks: An In-depth Review](https://mlsanity.com/paper/2311.14994/) (2023)
- [L2XGNN: Learning to Explain Graph Neural Networks](https://mlsanity.com/paper/2209.14402/) (2022)
- [Efficient GNN Explanation via Learning Removal-based Attribution](https://mlsanity.com/paper/2306.05760/) (2023)
- [Generative Explanations for Graph Neural Network: Methods and Evaluations](https://mlsanity.com/paper/2311.05764/) (2023)
- [When Graph Neural Network Meets Causality: Opportunities, Methodologies and An Outlook](https://mlsanity.com/paper/2312.12477/) (2023)
- [Towards Few-shot Self-explaining Graph Neural Networks](https://mlsanity.com/paper/2408.07340/) (2024)
- [Causal Temporal Graph Convolutional Neural Networks (CTGCN)](https://mlsanity.com/paper/2303.09634/) (2023)
- [Explainable Molecular Property Prediction: Aligning Chemical Concepts with Predictions via Language Models](https://mlsanity.com/paper/2405.16041/) (2024)
- [Unstructured and structured data: Can we have the best of both worlds with large language models?](https://mlsanity.com/paper/2304.13010/) (2023)
- [Causal State Distillation for Explainable Reinforcement Learning](https://mlsanity.com/paper/2401.00104/) (2023)
- [Predicting and Interpreting Energy Barriers of Metallic Glasses with Graph Neural Networks](https://mlsanity.com/paper/2401.08627/) (2023)
- [Graph-based Fake Account Detection: A Survey](https://mlsanity.com/paper/2507.06541/) (2025)
- [GRExplainer: A Universal Explanation Method for Temporal Graph Neural Networks](https://mlsanity.com/paper/2512.22772/) (2025)
- [Structure-Sensitive Graph Dictionary Embedding for Graph Classification](https://mlsanity.com/paper/2306.10505/) (2023)
- [ACGAN-GNNExplainer: Auxiliary Conditional Generative Explainer for Graph Neural Networks](https://mlsanity.com/paper/2309.16918/) (2023)
- [SynHING: Synthetic Heterogeneous Information Network Generation for Graph Learning and Explanation](https://mlsanity.com/paper/2401.04133/) (2024)

## Abstract (arXiv)

This paper proposes a new eXplanation framework, called OrphicX, for generating causal explanations for any graph neural networks (GNNs) based on learned latent causal factors. Specifically, we construct a distinct generative model and design an objective function that encourages the generative model to produce causal, compact, and faithful explanations. This is achieved by isolating the causal factors in the latent space of graphs by maximizing the information flow measurements. We theoretically analyze the cause-effect relationships in the proposed causal graph, identify node attributes as confounders between graphs and GNN predictions, and circumvent such confounder effect by leveraging the backdoor adjustment formula. Our framework is compatible with any GNNs, and it does not require access to the process by which the target GNN produces its predictions. In addition, it does not rely on the linear-independence assumption of the explained features, nor require prior knowledge on the graph learning tasks. We show a proof-of-concept of OrphicX on canonical classification problems on graph data. In particular, we analyze the explanatory subgraphs obtained from explanations for molecular graphs (i.e., Mutag) and quantitatively evaluate the explanation performance with frequently occurring subgraph patterns. Empirically, we show that OrphicX can effectively identify the causal semantics for generating causal explanations, significantly outperforming its alternatives.