# GradXKG: A Universal Explain-per-use Temporal Knowledge Graph Explainer

arXiv: https://arxiv.org/abs/2310.04889 · HTML: https://mlsanity.com/paper/2310.04889/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 23 of its 35 references that are in the map.
- First posted on arXiv 7 Oct 2023.

## Builds on (most cited first, 23 of 23)

- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Modeling Relational Data with Graph Convolutional Networks](https://mlsanity.com/paper/1703.06103/) (2017)
- [GNNExplainer: Generating Explanations for Graph Neural Networks](https://mlsanity.com/paper/1903.03894/) (2019)
- [Explainability in Graph Neural Networks: A Taxonomic Survey](https://mlsanity.com/paper/2012.15445/) (2020)
- [On Explainability of Graph Neural Networks via Subgraph Explorations](https://mlsanity.com/paper/2102.05152/) (2021)
- [PGM-Explainer: Probabilistic Graphical Model Explanations for Graph Neural Networks](https://mlsanity.com/paper/2010.05788/) (2020)
- [GraphLIME: Local Interpretable Model Explanations for Graph Neural Networks](https://mlsanity.com/paper/2001.06216/) (2020)
- [Know-Evolve: Deep Temporal Reasoning for Dynamic Knowledge Graphs](https://mlsanity.com/paper/1705.05742/) (2017)
- [Learning Sequence Encoders for Temporal Knowledge Graph Completion](https://mlsanity.com/paper/1809.03202/) (2018)
- [Higher-Order Explanations of Graph Neural Networks via Relevant Walks](https://mlsanity.com/paper/2006.03589/) (2020)
- [Temporal Knowledge Graph Reasoning Based on Evolutional Representation Learning](https://mlsanity.com/paper/2104.10353/) (2021)
- [Question Answering Over Temporal Knowledge Graphs](https://mlsanity.com/paper/2106.01515/) (2021)
- [Complex Temporal Question Answering on Knowledge Graphs](https://mlsanity.com/paper/2109.08935/) (2021)
- [TimeTraveler: Reinforcement Learning for Temporal Knowledge Graph Forecasting](https://mlsanity.com/paper/2109.04101/) (2021)
- [Temporal Knowledge Graph Forecasting with Neural ODE](https://mlsanity.com/paper/2101.05151/) (2021)
- [TLogic: Temporal Logical Rules for Explainable Link Forecasting on Temporal Knowledge Graphs](https://mlsanity.com/paper/2112.08025/) (2021)
- [EventKG: A Multilingual Event-Centric Temporal Knowledge Graph](https://mlsanity.com/paper/1804.04526/) (2018)
- [Temporal Knowledge Graph Reasoning with Historical Contrastive Learning](https://mlsanity.com/paper/2211.10904/) (2022)
- [Search from History and Reason for Future: Two-stage Reasoning on Temporal Knowledge Graphs](https://mlsanity.com/paper/2106.00327/) (2021)
- [Complex Evolutional Pattern Learning for Temporal Knowledge Graph Reasoning](https://mlsanity.com/paper/2203.07782/) (2022)
- [MetaTKG: Learning Evolutionary Meta-Knowledge for Temporal Knowledge Graph Reasoning](https://mlsanity.com/paper/2302.00893/) (2023)

## Abstract (arXiv)

Temporal knowledge graphs (TKGs) have shown promise for reasoning tasks by incorporating a temporal dimension to represent how facts evolve over time. However, existing TKG reasoning (TKGR) models lack explainability due to their black-box nature. Recent work has attempted to address this through customized model architectures that generate reasoning paths, but these recent approaches have limited generalizability and provide sparse explanatory output. To enable interpretability for most TKGR models, we propose GradXKG, a novel two-stage gradient-based approach for explaining Relational Graph Convolution Network (RGCN)-based TKGR models. First, a Grad-CAM-inspired RGCN explainer tracks gradients to quantify each node's contribution across timesteps in an efficient "explain-per-use" fashion. Second, an integrated gradients explainer consolidates importance scores for RGCN outputs, extending compatibility across diverse TKGR architectures based on RGCN. Together, the two explainers highlight the most critical nodes at each timestep for a given prediction. Our extensive experiments demonstrated that, by leveraging gradient information, GradXKG provides insightful explanations grounded in the model's logic in a timely manner for most RGCN-based TKGR models. This helps address the lack of interpretability in existing TKGR models and provides a universal explanation approach applicable across various models.