Interpreting Temporal Graph Neural Networks with Koopman Theory
Not on the map. Why?
Abstract (arXiv)
Spatiotemporal graph neural networks (STGNNs) have shown promising results in many domains, from forecasting to epidemiology. However, understanding the dynamics learned by these models and explaining their behaviour is significantly more difficult than for models that deal with static data. Inspired by Koopman theory, which allows a simple description of intricate, nonlinear dynamical systems, we introduce new explainability approaches for temporal graphs. Specifically, we present two methods to interpret the STGNN's decision process and identify the most relevant spatial and temporal patterns in the input for the task at hand. The first relies on dynamic mode decomposition (DMD), a Koopman-inspired dimensionality reduction method. The second relies on sparse identification of nonlinear dynamics (SINDy), a popular method for discovering governing equations of dynamical systems, which we use for the first time as a general tool for explainability. On semi-synthetic dissemination datasets, our methods correctly identify interpretable features such as the times at which infections occur and the infected nodes. We also validate the methods qualitatively on a real-world human motion dataset, where the explanations highlight the body parts most relevant for action recognition.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 38 of its 70 references that are in the map. First posted on arXiv 17 Oct 2024.
Builds on
38 of 70 references.
References · 38
- 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
- 2019A Comprehensive Survey on Graph Neural Networkscited by 2,906
- 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,348
- 2019Graph WaveNet for Deep Spatial-Temporal Graph Modelingcited by 820
- 2019GNNExplainer: Generating Explanations for Graph Neural Networkscited by 735
- 2017Deep learning for universal linear embeddings of nonlinear dynamicscited by 491
- 2020Parameterized Explainer for Graph Neural Networkcited by 423
- 2016Structured Sequence Modeling with Graph Convolutional Recurrent Networkscited by 273
- 2021Modern Koopman Theory for Dynamical Systemscited by 269
- 2015Structural-RNN: Deep Learning on Spatio-Temporal Graphscited by 234
- 2021CF-GNNExplainer: Counterfactual Explanations for Graph Neural Networkscited by 118
- 2021Towards a Rigorous Evaluation of Time-series Anomaly Detectioncited by 95
26 more
- 2022Evaluating Explainability for Graph Neural Networkscited by 85
- 2022Global Explainability of GNNs via Logic Combination of Learned Conceptscited by 64
- 2023Graph Neural Networks for temporal graphs: State of the art, open challenges, and opportunitiescited by 60
- 2022Scalable Spatiotemporal Graph Neural Networkscited by 52
- 2018Reservoir computing approaches for representation and classification of multivariate time seriescited by 38
- 2022GNNInterpreter: A Probabilistic Generative Model-Level Explanation for Graph Neural Networkscited by 34
- 2022Explaining the Explainers in Graph Neural Networks: a Comparative Studycited by 34
- 2023A Survey on Explainability of Graph Neural Networkscited by 28
- 2023From Continuous Dynamics to Graph Neural Networks: Neural Diffusion and Beyondcited by 26
- 2023Graph Deep Learning for Time Series Forecastingcited by 26
- 2024On Oversquashing in Graph Neural Networks Through the Lens of Dynamical Systemscited by 26
- 2022Sparse Graph Learning from Spatiotemporal Time Seriescited by 22
- 2023Graph-based Time Series Clustering for End-to-End Hierarchical Forecastingcited by 16
- 2021A Meta-Learning Approach for Training Explainable Graph Neural Networkscited by 15
- 2024Graph-based Forecasting with Missing Data through Spatiotemporal Downsamplingcited by 15
- 2019Temporal Graph Kernels for Classifying Dissemination Processescited by 14
- 2024Reconsidering Faithfulness in Regular, Self-Explainable and Domain Invariant GNNscited by 14
- 2023Explainable Spatio-Temporal Graph Neural Networkscited by 13
- 2024PyDMD: A Python package for robust dynamic mode decompositioncited by 13
- 2020Applications of Koopman Mode Analysis to Neural Networkscited by 12
- 2021An Operator Theoretic Approach for Analyzing Sequence Neural Networkscited by 12
- 2020GraphKKE: Graph Kernel Koopman Embedding for Human Microbiome Analysiscited by 3
- 2023Combining Stochastic Explainers and Subgraph Neural Networks can Increase Expressivity and Interpretabilitycited by 3
- 2022Understanding microbiome dynamics via interpretable graph representation learningcited by 2
- 2022Explainability in subgraphs-enhanced Graph Neural Networkscited by 2
- 2024When Graph Neural Networks Meet Dynamic Mode Decompositioncited by 2
Built on by
Citing papers per year: 2026: 2.
2 papers.