FlowX: Towards Explainable Graph Neural Networks via Message Flows
Not on the map. Why?
Abstract (arXiv)
We investigate the explainability of graph neural networks (GNNs) as a step toward elucidating their working mechanisms. While most current methods focus on explaining graph nodes, edges, or features, we argue that, as the inherent functional mechanism of GNNs, message flows are more natural for performing explainability. To this end, we propose a novel method here, known as FlowX, to explain GNNs by identifying important message flows. To quantify the importance of flows, we propose to follow the philosophy of Shapley values from cooperative game theory. To tackle the complexity of computing all coalitions' marginal contributions, we propose a flow sampling scheme to compute Shapley value approximations as initial assessments of further training. We then propose an information-controlled learning algorithm to train flow scores toward diverse explanation targets: necessary or sufficient explanations. Experimental studies on both synthetic and real-world datasets demonstrate that our proposed FlowX and its variants lead to improved explainability of GNNs. The code is available at https://github.com/divelab/DIG.
Key facts. Cited by 21 papers in the map (data through 14 Oct 2026). Builds on 50 of its 71 references that are in the map. First posted on arXiv 26 Jun 2022.
Builds on
50 of 71 references.
References · 50
- 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
- 2017Graph Attention Networkscited by 8,923
- 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,733
- 2018How Powerful are Graph Neural Networks?cited by 4,393
- 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,739
- 2015Learning Deep Features for Discriminative Localizationcited by 2,436
- 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,371
- 2019Simplifying Graph Convolutional Networkscited by 1,477
- 2017Learning Important Features Through Propagating Activation Differencescited by 1,458
- 2016Instance Normalization: The Missing Ingredient for Fast Stylizationcited by 1,198
- 2018Sanity Checks for Saliency Mapscited by 1,156
- 2017SmoothGrad: removing noise by adding noisecited by 1,109
38 more
- 2017MoleculeNet: A Benchmark for Molecular Machine Learningcited by 1,051
- 2020Simple and Deep Graph Convolutional Networkscited by 860
- 2018Hierarchical Graph Representation Learning with Differentiable Poolingcited by 787
- 2019GNNExplainer: Generating Explanations for Graph Neural Networkscited by 737
- 2017SchNet: A continuous-filter convolutional neural network for modeling quantum interactionscited by 571
- 2020Directional Message Passing for Molecular Graphscited by 561
- 2015Causal inference using invariant prediction: identification and confidence intervalscited by 536
- 2019DeepGCNs: Can GCNs Go as Deep as CNNs?cited by 527
- 2019Graph U-Netscited by 492
- 2020Parameterized Explainer for Graph Neural Networkcited by 426
- 2020Explainability in Graph Neural Networks: A Taxonomic Surveycited by 384
- 2018Graph Convolutional Policy Network for Goal-Directed Molecular Graph Generationcited by 364
- 2021On Explainability of Graph Neural Networks via Subgraph Explorationscited by 281
- 2018Learning to Explain: An Information-Theoretic Perspective on Model Interpretationcited by 248
- 2017Real Time Image Saliency for Black Box Classifierscited by 246
- 2021Invariance Principle Meets Information Bottleneck for Out-of-Distribution Generalizationcited by 220
- 2020XGNN: Towards Model-Level Explanations of Graph Neural Networkscited by 211
- 2020PGM-Explainer: Probabilistic Graphical Model Explanations for Graph Neural Networkscited by 211
- 2020GraphLIME: Local Interpretable Model Explanations for Graph Neural Networkscited by 209
- 2021GraphDF: A Discrete Flow Model for Molecular Graph Generationcited by 180
- 2022Interpretable and Generalizable Graph Learning via Stochastic Attention Mechanismcited by 171
- 2021Spherical Message Passing for 3D Graph Networkscited by 168
- 2019Explainability Techniques for Graph Convolutional Networkscited by 161
- 2020Interpreting Graph Neural Networks for NLP With Differentiable Edge Maskingcited by 153
- 2022Learning Causally Invariant Representations for Out-of-Distribution Generalization on Graphscited by 144
- 2022GOOD: A Graph Out-of-Distribution Benchmarkcited by 141
- 2020Higher-Order Explanations of Graph Neural Networks via Relevant Walkscited by 138
- 2021Generative Causal Explanations for Graph Neural Networkscited by 124
- 2021Size-Invariant Graph Representations for Graph Classification Extrapolationscited by 84
- 2021DIG: A Turnkey Library for Diving into Graph Deep Learning Researchcited by 76
- 2021Robust Counterfactual Explanations on Graph Neural Networkscited by 72
- 2021GraphSVX: Shapley Value Explanations for Graph Neural Networkscited by 64
- 2022GraphFramEx: Towards Systematic Evaluation of Explainability Methods for Graph Neural Networkscited by 58
- 2021Improving Subgraph Recognition with Variational Graph Information Bottleneckcited by 45
- 2022Reinforced Causal Explainer for Graph Neural Networkscited by 34
- 2019XFake: Explainable Fake News Detector with Visualizationscited by 32
- 2018Towards Explanation of DNN-based Prediction with Guided Feature Inversioncited by 31
- 2022Task-Agnostic Graph Explanationscited by 20
Built on by
Citing papers per year: 2022: 1; 2023: 3; 2024: 3; 2025: 7; 2026: 7.
21 papers.
Sort:
- 2023Joint Learning of Label and Environment Causal Independence for Graph Out-of-Distribution Generalizationcited by 29
- 2024GNNShap: Scalable and Accurate GNN Explanation using Shapley Valuescited by 12
- 2024Masked Graph Autoencoder with Non-discrete Bandwidthscited by 11
- 2023Graph Structure and Feature Extrapolation for Out-of-Distribution Generalizationcited by 10
- 2022L2XGNN: Learning to Explain Graph Neural Networkscited by 7
- 2024Generating Human Understandable Explanations for Node Embeddingscited by 2
- 2023Message-passing selection: Towards interpretable GNNs for graph classificationcited by 1
- 2025SliceGX: Layer-wise GNN Explanation with Model-slicingcited by 1
- 2025Explainable Graph Neural Networks via Structural Externalitiescited by 1
- 2025Fragment-Wise Interpretability in Graph Neural Networks via Molecule Decomposition and Contribution Analysiscited by 1
- 2025GRExplainer: A Universal Explanation Method for Temporal Graph Neural Networkscited by 1
- 2025Cooperative Causal GraphSAGEcited by 0
9 more
- 2025DistShap: Scalable GNN Explanations with Distributed Shapley Valuescited by 0
- 2025Model Metamers Reveal Invariances in Graph Neural Networkscited by 0
- 2026FSX: Message Flow Sensitivity Enhanced Structural Explainer for Graph Neural Networkscited by 0
- 2026Graph Neural Network-Informed Predictive Flows for Faster Ford-Fulkerson and PAC-Learnabilitycited by 0
- 2026Graph Navier Stokes Networkscited by 0
- 2026Towards the Explainability of Temporal Graph Networks via Memory Backtracking and Topological Attributioncited by 0
- 2026A Polynomial Architecture-Attribution Co-Design Framework for Exact Aumann-Shapley Attribution in GNNscited by 0
- 2026Self-Explainable Multi-Label Graph Neural Network for Correlated Evidence Attributioncited by 0
- 2026Beyond Trained Models: Compiling GNNs for a Sound Explainer Benchmarkcited by 0
- 2026Beyond Trained Models: Compiling GNNs for a Sound Explainer Benchmark2 Oct 2026
- 2026Self-Explainable Multi-Label Graph Neural Network for Correlated Evidence Attribution27 Aug 2026
- 2026A Polynomial Architecture-Attribution Co-Design Framework for Exact Aumann-Shapley Attribution in GNNs23 Jul 2026
- 2026Towards the Explainability of Temporal Graph Networks via Memory Backtracking and Topological Attribution4 Jul 2026
- 2026Graph Navier Stokes Networks20 May 2026
- 2026Graph Neural Network-Informed Predictive Flows for Faster Ford-Fulkerson and PAC-Learnability23 Apr 2026
- 2026FSX: Message Flow Sensitivity Enhanced Structural Explainer for Graph Neural Networks21 Jan 2026
- 2025GRExplainer: A Universal Explanation Method for Temporal Graph Neural Networks28 Dec 2025
- 2025Model Metamers Reveal Invariances in Graph Neural Networks20 Oct 2025
- 2025Fragment-Wise Interpretability in Graph Neural Networks via Molecule Decomposition and Contribution Analysis20 Aug 2025
- 2025Explainable Graph Neural Networks via Structural Externalities19 Jul 2025
- 2025DistShap: Scalable GNN Explanations with Distributed Shapley Values27 Jun 2025
9 more
- 2025SliceGX: Layer-wise GNN Explanation with Model-slicing22 Jun 2025
- 2025Cooperative Causal GraphSAGE20 May 2025
- 2024Generating Human Understandable Explanations for Node Embeddings11 Jun 2024
- 2024Masked Graph Autoencoder with Non-discrete Bandwidths6 Feb 2024
- 2024GNNShap: Scalable and Accurate GNN Explanation using Shapley Values9 Jan 2024
- 2023Graph Structure and Feature Extrapolation for Out-of-Distribution Generalization13 Jun 2023
- 2023Message-passing selection: Towards interpretable GNNs for graph classification3 Jun 2023
- 2023Joint Learning of Label and Environment Causal Independence for Graph Out-of-Distribution Generalization1 Jun 2023
- 2022L2XGNN: Learning to Explain Graph Neural Networks28 Sep 2022
- 2023Joint Learning of Label and Environment Causal Independence for Graph Out-of-Distribution Generalizationcited by 29
- 2024GNNShap: Scalable and Accurate GNN Explanation using Shapley Valuescited by 12
- 2024Masked Graph Autoencoder with Non-discrete Bandwidthscited by 11
- 2023Graph Structure and Feature Extrapolation for Out-of-Distribution Generalizationcited by 10
- 2022L2XGNN: Learning to Explain Graph Neural Networkscited by 7
- 2024Generating Human Understandable Explanations for Node Embeddingscited by 2
- 2023Message-passing selection: Towards interpretable GNNs for graph classificationcited by 1
- 2025SliceGX: Layer-wise GNN Explanation with Model-slicingcited by 1
- 2025Explainable Graph Neural Networks via Structural Externalitiescited by 1
- 2025Fragment-Wise Interpretability in Graph Neural Networks via Molecule Decomposition and Contribution Analysiscited by 1
- 2025GRExplainer: A Universal Explanation Method for Temporal Graph Neural Networkscited by 1
- 2025Cooperative Causal GraphSAGEcited by 0
12 most cited of 21 All 21 (JSON)
- 2026Beyond Trained Models: Compiling GNNs for a Sound Explainer Benchmark2 Oct 2026
- 2026Self-Explainable Multi-Label Graph Neural Network for Correlated Evidence Attribution27 Aug 2026
- 2026A Polynomial Architecture-Attribution Co-Design Framework for Exact Aumann-Shapley Attribution in GNNs23 Jul 2026
- 2026Towards the Explainability of Temporal Graph Networks via Memory Backtracking and Topological Attribution4 Jul 2026
- 2026Graph Navier Stokes Networks20 May 2026
- 2026Graph Neural Network-Informed Predictive Flows for Faster Ford-Fulkerson and PAC-Learnability23 Apr 2026
- 2026FSX: Message Flow Sensitivity Enhanced Structural Explainer for Graph Neural Networks21 Jan 2026
- 2025GRExplainer: A Universal Explanation Method for Temporal Graph Neural Networks28 Dec 2025
- 2025Model Metamers Reveal Invariances in Graph Neural Networks20 Oct 2025
- 2025Fragment-Wise Interpretability in Graph Neural Networks via Molecule Decomposition and Contribution Analysis20 Aug 2025
- 2025Explainable Graph Neural Networks via Structural Externalities19 Jul 2025
- 2025DistShap: Scalable GNN Explanations with Distributed Shapley Values27 Jun 2025
12 newest of 21 All 21 (JSON)