# GNNInterpreter: A Probabilistic Generative Model-Level Explanation for Graph Neural Networks

arXiv: https://arxiv.org/abs/2209.07924 · HTML: https://mlsanity.com/paper/2209.07924/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 43 references that are in the map.
- First posted on arXiv 15 Sep 2022.

## Builds on (most cited first, 28 of 28)

- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [How Powerful are Graph Neural Networks?](https://mlsanity.com/paper/1810.00826/) (2018)
- [Neural Message Passing for Quantum Chemistry](https://mlsanity.com/paper/1704.01212/) (2017)
- [A Comprehensive Survey on Graph Neural Networks](https://mlsanity.com/paper/1901.00596/) (2019)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering](https://mlsanity.com/paper/1606.09375/) (2016)
- [The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables](https://mlsanity.com/paper/1611.00712/) (2016)
- [Link Prediction Based on Graph Neural Networks](https://mlsanity.com/paper/1802.09691/) (2018)
- [Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Images](https://mlsanity.com/paper/1412.1897/) (2014)
- [GNNExplainer: Generating Explanations for Graph Neural Networks](https://mlsanity.com/paper/1903.03894/) (2019)
- [Parameterized Explainer for Graph Neural Network](https://mlsanity.com/paper/2011.04573/) (2020)
- [Explainability in Graph Neural Networks: A Taxonomic Survey](https://mlsanity.com/paper/2012.15445/) (2020)
- [Top-down Neural Attention by Excitation Backprop](https://mlsanity.com/paper/1608.00507/) (2016)
- [On Explainability of Graph Neural Networks via Subgraph Explorations](https://mlsanity.com/paper/2102.05152/) (2021)
- [XGNN: Towards Model-Level Explanations of Graph Neural Networks](https://mlsanity.com/paper/2006.02587/) (2020)
- [PGM-Explainer: Probabilistic Graphical Model Explanations for Graph Neural Networks](https://mlsanity.com/paper/2010.05788/) (2020)
- [GraphLIME: Local Interpretable Model Explanations for Graph Neural Networks](https://mlsanity.com/paper/2001.06216/) (2020)
- [Layer-wise Relevance Propagation for Neural Networks with Local Renormalization Layers](https://mlsanity.com/paper/1604.00825/) (2016)
- [Explainability Techniques for Graph Convolutional Networks](https://mlsanity.com/paper/1905.13686/) (2019)
- [Generative Causal Explanations for Graph Neural Networks](https://mlsanity.com/paper/2104.06643/) (2021)
- [GraphSVX: Shapley Value Explanations for Graph Neural Networks](https://mlsanity.com/paper/2104.10482/) (2021)
- [OrphicX: A Causality-Inspired Latent Variable Model for Interpreting Graph Neural Networks](https://mlsanity.com/paper/2203.15209/) (2022)
- [Layerwise Relevance Visualization in Convolutional Text Graph Classifiers](https://mlsanity.com/paper/1909.10911/) (2019)
- [MotifExplainer: a Motif-based Graph Neural Network Explainer](https://mlsanity.com/paper/2202.00519/) (2022)

## Built on by (most cited first, 30 of 34)

- [Explaining the Explainers in Graph Neural Networks: a Comparative Study](https://mlsanity.com/paper/2210.15304/) (2022)
- [A Survey on Explainability of Graph Neural Networks](https://mlsanity.com/paper/2306.01958/) (2023)
- [TempME: Towards the Explainability of Temporal Graph Neural Networks via Motif Discovery](https://mlsanity.com/paper/2310.19324/) (2023)
- [Graph Neural Network Explanations are Fragile](https://mlsanity.com/paper/2406.03193/) (2024)
- [Towards Robust Fidelity for Evaluating Explainability of Graph Neural Networks](https://mlsanity.com/paper/2310.01820/) (2023)
- [No Metric to Rule Them All: Toward Principled Evaluations of Graph-Learning Datasets](https://mlsanity.com/paper/2502.02379/) (2025)
- [Explaining Graph Neural Networks via Structure-aware Interaction Index](https://mlsanity.com/paper/2405.14352/) (2024)
- [D4Explainer: In-Distribution GNN Explanations via Discrete Denoising Diffusion](https://mlsanity.com/paper/2310.19321/) (2023)
- [Graph Neural Network Causal Explanation via Neural Causal Models](https://mlsanity.com/paper/2407.09378/) (2024)
- [MAGE: Model-Level Graph Neural Networks Explanations via Motif-based Graph Generation](https://mlsanity.com/paper/2405.12519/) (2024)
- [A Self-Explainable Heterogeneous GNN for Relational Deep Learning](https://mlsanity.com/paper/2412.00521/) (2024)
- [Recent Advances in Malware Detection: Graph Learning and Explainability](https://mlsanity.com/paper/2502.10556/) (2025)
- [Generative Explanations for Graph Neural Network: Methods and Evaluations](https://mlsanity.com/paper/2311.05764/) (2023)
- [Towards Few-shot Self-explaining Graph Neural Networks](https://mlsanity.com/paper/2408.07340/) (2024)
- [xAI-Drop: Don't Use What You Cannot Explain](https://mlsanity.com/paper/2407.20067/) (2024)
- [Unstructured and structured data: Can we have the best of both worlds with large language models?](https://mlsanity.com/paper/2304.13010/) (2023)
- [Everybody Needs a Little HELP: Explaining Graphs via Hierarchical Concepts](https://mlsanity.com/paper/2311.15112/) (2023)
- [Interpreting Temporal Graph Neural Networks with Koopman Theory](https://mlsanity.com/paper/2410.13469/) (2024)
- [GnnXemplar: Exemplars to Explanations -- Natural Language Rules for Global GNN Interpretability](https://mlsanity.com/paper/2509.18376/) (2025)
- [KS-GNNExplainer: Global Model Interpretation Through Instance Explanations On Histopathology images](https://mlsanity.com/paper/2304.08240/) (2023)
- [Discovering robust biomarkers of psychiatric disorders from resting-state functional MRI via graph neural networks: A systematic review](https://mlsanity.com/paper/2405.00577/) (2024)
- [LogicXGNN: Grounded Logical Rules for Explaining Graph Neural Networks](https://mlsanity.com/paper/2503.19476/) (2025)
- [Towards Faithful Class-level Self-explainability in Graph Neural Networks by Subgraph Dependencies](https://mlsanity.com/paper/2508.11513/) (2025)
- [Relevant Walk Search for Explaining Graph Neural Networks](https://mlsanity.com/paper/2605.23673/) (2026)
- [PAC Learnability under Explanation-Preserving Graph Perturbations](https://mlsanity.com/paper/2402.05039/) (2024)
- [SES: Bridging the Gap Between Explainability and Prediction of Graph Neural Networks](https://mlsanity.com/paper/2407.11358/) (2024)
- [PAGE: Parametric Generative Explainer for Graph Neural Network](https://mlsanity.com/paper/2408.14042/) (2024)
- [GIN-Graph: A Generative Interpretation Network for Model-Level Explanation of Graph Neural Networks](https://mlsanity.com/paper/2503.06352/) (2025)
- [Model-Level GNN Explanations via Rule-to-Graph Readout for Logit Reconstruction](https://mlsanity.com/paper/2503.09051/) (2025)
- [How do Probabilistic Graphical Models and Graph Neural Networks Look at Network Data?](https://mlsanity.com/paper/2506.11869/) (2025)

## Abstract (arXiv)

Recently, Graph Neural Networks (GNNs) have significantly advanced the performance of machine learning tasks on graphs. However, this technological breakthrough makes people wonder: how does a GNN make such decisions, and can we trust its prediction with high confidence? When it comes to some critical fields, such as biomedicine, where making wrong decisions can have severe consequences, it is crucial to interpret the inner working mechanisms of GNNs before applying them. In this paper, we propose a model-agnostic model-level explanation method for different GNNs that follow the message passing scheme, GNNInterpreter, to explain the high-level decision-making process of the GNN model. More specifically, GNNInterpreter learns a probabilistic generative graph distribution that produces the most discriminative graph pattern the GNN tries to detect when making a certain prediction by optimizing a novel objective function specifically designed for the model-level explanation for GNNs. Compared to existing works, GNNInterpreter is more flexible and computationally efficient in generating explanation graphs with different types of node and edge features, without introducing another blackbox or requiring manually specified domain-specific rules. In addition, the experimental studies conducted on four different datasets demonstrate that the explanation graphs generated by GNNInterpreter match the desired graph pattern if the model is ideal; otherwise, potential model pitfalls can be revealed by the explanation. The official implementation can be found at https://github.com/yolandalalala/GNNInterpreter.