# Transformer-based Reasoning for Learning Evolutionary Chain of Events on Temporal Knowledge Graph

arXiv: https://arxiv.org/abs/2405.00352 · HTML: https://mlsanity.com/paper/2405.00352/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 38 references that are in the map.
- First posted on arXiv 1 May 2024.

## Builds on (most cited first, 17 of 17)

- [Modeling Relational Data with Graph Convolutional Networks](https://mlsanity.com/paper/1703.06103/) (2017)
- [MLP-Mixer: An all-MLP Architecture for Vision](https://mlsanity.com/paper/2105.01601/) (2021)
- [Are Graph Augmentations Necessary? Simple Graph Contrastive Learning for Recommendation](https://mlsanity.com/paper/2112.08679/) (2021)
- [Tensor Decompositions for temporal knowledge base completion](https://mlsanity.com/paper/2004.04926/) (2020)
- [Temporal Knowledge Graph Reasoning Based on Evolutional Representation Learning](https://mlsanity.com/paper/2104.10353/) (2021)
- [BoxE: A Box Embedding Model for Knowledge Base Completion](https://mlsanity.com/paper/2007.06267/) (2020)
- [Diachronic Embedding for Temporal Knowledge Graph Completion](https://mlsanity.com/paper/1907.03143/) (2019)
- [TimeTraveler: Reinforcement Learning for Temporal Knowledge Graph Forecasting](https://mlsanity.com/paper/2109.04101/) (2021)
- [TLogic: Temporal Logical Rules for Explainable Link Forecasting on Temporal Knowledge Graphs](https://mlsanity.com/paper/2112.08025/) (2021)
- [TeRo: A Time-aware Knowledge Graph Embedding via Temporal Rotation](https://mlsanity.com/paper/2010.01029/) (2020)
- [Search from History and Reason for Future: Two-stage Reasoning on Temporal Knowledge Graphs](https://mlsanity.com/paper/2106.00327/) (2021)
- [ChronoR: Rotation Based Temporal Knowledge Graph Embedding](https://mlsanity.com/paper/2103.10379/) (2021)
- [Temporal Knowledge Graph Completion using Box Embeddings](https://mlsanity.com/paper/2109.08970/) (2021)
- [Temporal Knowledge Base Completion: New Algorithms and Evaluation Protocols](https://mlsanity.com/paper/2005.05035/) (2020)
- [DyERNIE: Dynamic Evolution of Riemannian Manifold Embeddings for Temporal Knowledge Graph Completion](https://mlsanity.com/paper/2011.03984/) (2020)
- [Learning to Sample and Aggregate: Few-shot Reasoning over Temporal Knowledge Graphs](https://mlsanity.com/paper/2210.08654/) (2022)
- [Transformer-based Entity Typing in Knowledge Graphs](https://mlsanity.com/paper/2210.11151/) (2022)

## Built on by (most cited first, 8 of 8)

- [Video-CoE: Reinforcing Video Event Prediction via Chain of Events](https://mlsanity.com/paper/2603.14935/) (2026)
- [UniHR: Hierarchical Representation Learning for Unified Knowledge Graph Link Prediction](https://mlsanity.com/paper/2411.07019/) (2024)
- [Video-Language Alignment via Spatio-Temporal Graph Transformer](https://mlsanity.com/paper/2407.11677/) (2024)
- [DynaGen: Unifying Temporal Knowledge Graph Reasoning with Dynamic Subgraphs and Generative Regularization](https://mlsanity.com/paper/2512.12669/) (2025)
- [CID-TKG: Collaborative Historical Invariance and Evolutionary Dynamics Learning for Temporal Knowledge Graph Reasoning](https://mlsanity.com/paper/2604.09600/) (2026)
- [CHE-TKG: Collaborative Historical Evidence and Evolutionary Dynamics Learning for Temporal Knowledge Graph Reasoning](https://mlsanity.com/paper/2605.04652/) (2026)
- [AdaTKG: Adaptive Memory for Temporal Knowledge Graph Reasoning](https://mlsanity.com/paper/2605.07121/) (2026)
- [GAttNHP: Group Attention Neural Hawkes Process for Extrapolation Reasoning in Temporal Knowledge Graphs](https://mlsanity.com/paper/2607.14733/) (2026)

## Abstract (arXiv)

Temporal Knowledge Graph (TKG) reasoning often involves completing missing factual elements along the timeline. Although existing methods can learn good embeddings for each factual element in quadruples by integrating temporal information, they often fail to infer the evolution of temporal facts. This is mainly because of (1) insufficiently exploring the internal structure and semantic relationships within individual quadruples and (2) inadequately learning a unified representation of the contextual and temporal correlations among different quadruples. To overcome these limitations, we propose a novel Transformer-based reasoning model (dubbed ECEformer) for TKG to learn the Evolutionary Chain of Events (ECE). Specifically, we unfold the neighborhood subgraph of an entity node in chronological order, forming an evolutionary chain of events as the input for our model. Subsequently, we utilize a Transformer encoder to learn the embeddings of intra-quadruples for ECE. We then craft a mixed-context reasoning module based on the multi-layer perceptron (MLP) to learn the unified representations of inter-quadruples for ECE while accomplishing temporal knowledge reasoning. In addition, to enhance the timeliness of the events, we devise an additional time prediction task to complete effective temporal information within the learned unified representation. Extensive experiments on six benchmark datasets verify the state-of-the-art performance and the effectiveness of our method.