# Layerwise Relevance Visualization in Convolutional Text Graph Classifiers

arXiv: https://arxiv.org/abs/1909.10911 · HTML: https://mlsanity.com/paper/1909.10911/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Sep 2019.

## Built on by (most cited first, 30 of 32)

- [On Explainability of Graph Neural Networks via Subgraph Explorations](https://mlsanity.com/paper/2102.05152/) (2021)
- [Benchmarking and Survey of Explanation Methods for Black Box Models](https://mlsanity.com/paper/2102.13076/) (2021)
- [A Comprehensive Survey on Trustworthy Graph Neural Networks: Privacy, Robustness, Fairness, and Explainability](https://mlsanity.com/paper/2204.08570/) (2022)
- [ProtGNN: Towards Self-Explaining Graph Neural Networks](https://mlsanity.com/paper/2112.00911/) (2021)
- [Graph-Based Deep Learning for Medical Diagnosis and Analysis: Past, Present and Future](https://mlsanity.com/paper/2105.13137/) (2021)
- [A Survey on Graph-Based Deep Learning for Computational Histopathology](https://mlsanity.com/paper/2107.00272/) (2021)
- [Explaining the Explainers in Graph Neural Networks: a Comparative Study](https://mlsanity.com/paper/2210.15304/) (2022)
- [GNNInterpreter: A Probabilistic Generative Model-Level Explanation for Graph Neural Networks](https://mlsanity.com/paper/2209.07924/) (2022)
- [HistoCartography: A Toolkit for Graph Analytics in Digital Pathology](https://mlsanity.com/paper/2107.10073/) (2021)
- [A Survey of Trustworthy Graph Learning: Reliability, Explainability, and Privacy Protection](https://mlsanity.com/paper/2205.10014/) (2022)
- [DEGREE: Decomposition Based Explanation For Graph Neural Networks](https://mlsanity.com/paper/2305.12895/) (2023)
- [Dynamic Causal Explanation Based Diffusion-Variational Graph Neural Network for Spatio-temporal Forecasting](https://mlsanity.com/paper/2305.09703/) (2023)
- [GNNShap: Scalable and Accurate GNN Explanation using Shapley Values](https://mlsanity.com/paper/2401.04829/) (2024)
- [GNNX-BENCH: Unravelling the Utility of Perturbation-based GNN Explainers through In-depth Benchmarking](https://mlsanity.com/paper/2310.01794/) (2023)
- [Towards Explainable Motion Prediction using Heterogeneous Graph Representations](https://mlsanity.com/paper/2212.03806/) (2022)
- [Defending Against Backdoor Attack on Graph Nerual Network by Explainability](https://mlsanity.com/paper/2209.02902/) (2022)
- [Graph Neural Networks based Log Anomaly Detection and Explanation](https://mlsanity.com/paper/2307.00527/) (2023)
- [View-based Explanations for Graph Neural Networks](https://mlsanity.com/paper/2401.02086/) (2024)
- [L2XGNN: Learning to Explain Graph Neural Networks](https://mlsanity.com/paper/2209.14402/) (2022)
- [MEGAN: Multi-Explanation Graph Attention Network](https://mlsanity.com/paper/2211.13236/) (2022)
- [GraphNarrator: Generating Textual Explanations for Graph Neural Networks](https://mlsanity.com/paper/2410.15268/) (2024)
- [ENGAGE: Explanation Guided Data Augmentation for Graph Representation Learning](https://mlsanity.com/paper/2307.01053/) (2023)
- [Explainable Graph Neural Networks Under Fire](https://mlsanity.com/paper/2406.06417/) (2024)
- [Towards the Explanation of Graph Neural Networks in Digital Pathology with Information Flows](https://mlsanity.com/paper/2112.09895/) (2021)
- [Generating Robust Counterfactual Witnesses for Graph Neural Networks](https://mlsanity.com/paper/2404.19519/) (2024)
- [Semantic Interpretation and Validation of Graph Attention-based Explanations for GNN Models](https://mlsanity.com/paper/2308.04220/) (2023)
- [KS-GNNExplainer: Global Model Interpretation Through Instance Explanations On Histopathology images](https://mlsanity.com/paper/2304.08240/) (2023)
- [Database Views as Explanations for Relational Deep Learning](https://mlsanity.com/paper/2509.09482/) (2025)
- [Two Birds with One Stone: Enhancing Uncertainty Quantification and Interpretability with Graph Functional Neural Process](https://mlsanity.com/paper/2508.17097/) (2025)
- [ProteinPNet: Prototypical Part Networks for Concept Learning in Spatial Proteomics](https://mlsanity.com/paper/2512.02983/) (2025)

## Abstract (arXiv)

Representations in the hidden layers of Deep Neural Networks (DNN) are often hard to interpret since it is difficult to project them into an interpretable domain. Graph Convolutional Networks (GCN) allow this projection, but existing explainability methods do not exploit this fact, i.e. do not focus their explanations on intermediate states. In this work, we present a novel method that traces and visualizes features that contribute to a classification decision in the visible and hidden layers of a GCN. Our method exposes hidden cross-layer dynamics in the input graph structure. We experimentally demonstrate that it yields meaningful layerwise explanations for a GCN sentence classifier.