mlsanityData to 7 Oct 2026

Higher-Order Explanations of Graph Neural Networks via Relevant Walks

arXiv 2006.03589 ↗First posted 5 Jun 2020cs.LG cs.AI stat.MLcited by 138

Not on the map. Why?

Abstract (arXiv)

Graph Neural Networks (GNNs) are a popular approach for predicting graph structured data. As GNNs tightly entangle the input graph into the neural network structure, common explainable AI approaches are not applicable. To a large extent, GNNs have remained black-boxes for the user so far. In this paper, we show that GNNs can in fact be naturally explained using higher-order expansions, i.e. by identifying groups of edges that jointly contribute to the prediction. Practically, we find that such explanations can be extracted using a nested attribution scheme, where existing techniques such as layer-wise relevance propagation (LRP) can be applied at each step. The output is a collection of walks into the input graph that are relevant for the prediction. Our novel explanation method, which we denote by GNN-LRP, is applicable to a broad range of graph neural networks and lets us extract practically relevant insights on sentiment analysis of text data, structure-property relationships in quantum chemistry, and image classification.

Key facts. Cited by 138 papers in the map (data through 7 Oct 2026). First posted on arXiv 5 Jun 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 15; 2022: 29; 2023: 27; 2024: 32; 2025: 20; 2026: 15.

138 papers.

Sort:

By area:

  1. 2022From Attribution Maps to Human-Understandable Explanations through Concept Relevance Propagationcited by 141
  2. 2022A Comprehensive Survey on Trustworthy Graph Neural Networks: Privacy, Robustness, Fairness, and Explainabilitycited by 118
  3. 2022XAI for Transformers: Better Explanations through Conservative Propagationcited by 100
  4. 2022Trustworthy Graph Neural Networks: Aspects, Methods and Trendscited by 88
  5. 2021DIG: A Turnkey Library for Diving into Graph Deep Learning Researchcited by 76
  6. 2022GraphFramEx: Towards Systematic Evaluation of Explainability Methods for Graph Neural Networkscited by 57
  7. 2023Towards Self-Interpretable Graph-Level Anomaly Detectioncited by 41
  8. 2021Probing GNN Explainers: A Rigorous Theoretical and Empirical Analysis of GNN Explanation Methodscited by 36
  9. 2021GCExplainer: Human-in-the-Loop Concept-based Explanations for Graph Neural Networkscited by 35
  10. 2021Zorro: Valid, Sparse, and Stable Explanations in Graph Neural Networkscited by 34
  11. 2022Reinforced Causal Explainer for Graph Neural Networkscited by 34
  12. 2022Explaining the Explainers in Graph Neural Networks: a Comparative Studycited by 34
38 more
  1. 2024MambaLRP: Explaining Selective State Space Sequence Modelscited by 31
  2. 2021Toward Explainable AI for Regression Modelscited by 30
  3. 2023A Survey on Explainability of Graph Neural Networkscited by 28
  4. 2023DAG Matters! GFlowNets Enhanced Explainer For Graph Neural Networkscited by 27
  5. 2023TempME: Towards the Explainability of Temporal Graph Neural Networks via Motif Discoverycited by 27
  6. 2022Higher-order Clustering and Pooling for Graph Neural Networkscited by 26
  7. 2022Disentangled Explanations of Neural Network Predictions by Finding Relevant Subspacescited by 25
  8. 2024The Clever Hans Effect in Unsupervised Learningcited by 24
  9. 2022FlowX: Towards Explainable Graph Neural Networks via Message Flowscited by 20
  10. 2023DEGREE: Decomposition Based Explanation For Graph Neural Networkscited by 18
  11. 2021PredDiff: Explanations and Interactions from Conditional Expectationscited by 16
  12. 2024xMIL: Insightful Explanations for Multiple Instance Learning in Histopathologycited by 15
  13. 2022ES-GNN: Generalizing Graph Neural Networks Beyond Homophily with Edge Splittingcited by 13
  14. 2022Towards Faithful and Consistent Explanations for Graph Neural Networkscited by 13
  15. 2021Algorithmic Concept-based Explainable Reasoningcited by 12
  16. 2021Explaining machine-learned particle-flow reconstructioncited by 12
  17. 2022An Explainer for Temporal Graph Neural Networkscited by 12
  18. 2024GNNShap: Scalable and Accurate GNN Explanation using Shapley Valuescited by 12
  19. 2024Graph Neural Networks for Vulnerability Detection: A Counterfactual Explanationcited by 12
  20. 2025Exact Computation of Any-Order Shapley Interactions for Graph Neural Networkscited by 12
  21. 2022MotifExplainer: a Motif-based Graph Neural Network Explainercited by 11
  22. 2023GNNX-BENCH: Unravelling the Utility of Perturbation-based GNN Explainers through In-depth Benchmarkingcited by 11
  23. 2021An Empirical Study: Extensive Deep Temporal Point Processcited by 10
  24. 2022Deconfounding to Explanation Evaluation in Graph Neural Networkscited by 10
  25. 2022Towards Explainable Motion Prediction using Heterogeneous Graph Representationscited by 10
  26. 2024GOAt: Explaining Graph Neural Networks via Graph Output Attributioncited by 10
  27. 2024Explaining Graph Neural Networks via Structure-aware Interaction Indexcited by 10
  28. 2025RelP: Faithful and Efficient Circuit Discovery in Language Models via Relevance Patchingcited by 10
  29. 2022Defending Against Backdoor Attack on Graph Nerual Network by Explainabilitycited by 9
  30. 2023Preemptively Pruning Clever-Hans Strategies in Deep Neural Networkscited by 9
  31. 2023D4Explainer: In-Distribution GNN Explanations via Discrete Denoising Diffusioncited by 9
  32. 2024Explaining Predictive Uncertainty by Exposing Second-Order Effectscited by 9
  33. 2024Explaining Text Similarity in Transformer Modelscited by 9
  34. 2025Capturing Polysemanticity with PRISM: A Multi-Concept Feature Description Frameworkcited by 9
  35. 2021Graph Denoising with Framelet Regularizercited by 8
  36. 2023Rethinking Explaining Graph Neural Networks via Non-parametric Subgraph Matchingcited by 8
  37. 2023Exploring Causal Learning through Graph Neural Networks: An In-depth Reviewcited by 8
  38. 2024Higher-order Spatio-temporal Physics-incorporated Graph Neural Network for Multivariate Time Series Imputationcited by 8

50 most cited of 138 All 138 (JSON)