# Temporal Knowledge Graph Forecasting with Neural ODE

arXiv: https://arxiv.org/abs/2101.05151 · HTML: https://mlsanity.com/paper/2101.05151/

## Key facts

- Cited by 61 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 40 references that are in the map.
- First posted on arXiv 13 Jan 2021.

## Builds on (most cited first, 24 of 24)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Neural Message Passing for Quantum Chemistry](https://mlsanity.com/paper/1704.01212/) (2017)
- [Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering](https://mlsanity.com/paper/1606.09375/) (2016)
- [Modeling Relational Data with Graph Convolutional Networks](https://mlsanity.com/paper/1703.06103/) (2017)
- [Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition](https://mlsanity.com/paper/1801.07455/) (2018)
- [Embedding Entities and Relations for Learning and Inference in Knowledge Bases](https://mlsanity.com/paper/1412.6575/) (2014)
- [Composition-based Multi-Relational Graph Convolutional Networks](https://mlsanity.com/paper/1911.03082/) (2019)
- [Learning Convolutional Neural Networks for Graphs](https://mlsanity.com/paper/1605.05273/) (2016)
- [TuckER: Tensor Factorization for Knowledge Graph Completion](https://mlsanity.com/paper/1901.09590/) (2019)
- [Go for a Walk and Arrive at the Answer: Reasoning Over Paths in Knowledge Bases using Reinforcement Learning](https://mlsanity.com/paper/1711.05851/) (2017)
- [Know-Evolve: Deep Temporal Reasoning for Dynamic Knowledge Graphs](https://mlsanity.com/paper/1705.05742/) (2017)
- [Learning Sequence Encoders for Temporal Knowledge Graph Completion](https://mlsanity.com/paper/1809.03202/) (2018)
- [Beta Embeddings for Multi-Hop Logical Reasoning in Knowledge Graphs](https://mlsanity.com/paper/2010.11465/) (2020)
- [Tensor Decompositions for temporal knowledge base completion](https://mlsanity.com/paper/2004.04926/) (2020)
- [Diachronic Embedding for Temporal Knowledge Graph Completion](https://mlsanity.com/paper/1907.03143/) (2019)
- [Learning from History: Modeling Temporal Knowledge Graphs with Sequential Copy-Generation Networks](https://mlsanity.com/paper/2012.08492/) (2020)
- [DyERNIE: Dynamic Evolution of Riemannian Manifold Embeddings for Temporal Knowledge Graph Completion](https://mlsanity.com/paper/2011.03984/) (2020)
- [xERTE: Explainable Reasoning on Temporal Knowledge Graphs for Forecasting Future Links](https://mlsanity.com/paper/2012.15537/) (2020)
- [Graph Hawkes Neural Network for Forecasting on Temporal Knowledge Graphs](https://mlsanity.com/paper/2003.13432/) (2020)
- [Interpolation Technique to Speed Up Gradients Propagation in Neural ODEs](https://mlsanity.com/paper/2003.05271/) (2020)
- [Temporal Graph Modeling for Skeleton-based Action Recognition](https://mlsanity.com/paper/2012.08804/) (2020)

## Built on by (most cited first, 30 of 61)

- [Large Models for Time Series and Spatio-Temporal Data: A Survey and Outlook](https://mlsanity.com/paper/2310.10196/) (2023)
- [A Survey of Knowledge Graph Reasoning on Graph Types: Static, Dynamic, and Multimodal](https://mlsanity.com/paper/2212.05767/) (2022)
- [Temporal Knowledge Graph Completion: A Survey](https://mlsanity.com/paper/2201.08236/) (2022)
- [Temporal Knowledge Graph Reasoning with Historical Contrastive Learning](https://mlsanity.com/paper/2211.10904/) (2022)
- [MIRAI: Evaluating LLM Agents for Event Forecasting](https://mlsanity.com/paper/2407.01231/) (2024)
- [Temporal Knowledge Graph Forecasting Without Knowledge Using In-Context Learning](https://mlsanity.com/paper/2305.10613/) (2023)
- [TGB 2.0: A Benchmark for Learning on Temporal Knowledge Graphs and Heterogeneous Graphs](https://mlsanity.com/paper/2406.09639/) (2024)
- [Complex Evolutional Pattern Learning for Temporal Knowledge Graph Reasoning](https://mlsanity.com/paper/2203.07782/) (2022)
- [GenTKG: Generative Forecasting on Temporal Knowledge Graph with Large Language Models](https://mlsanity.com/paper/2310.07793/) (2023)
- [A Survey on Temporal Knowledge Graph: Representation Learning and Applications](https://mlsanity.com/paper/2403.04782/) (2024)
- [Learning to Sample and Aggregate: Few-shot Reasoning over Temporal Knowledge Graphs](https://mlsanity.com/paper/2210.08654/) (2022)
- [History repeats Itself: A Baseline for Temporal Knowledge Graph Forecasting](https://mlsanity.com/paper/2404.16726/) (2024)
- [A Survey on Temporal Knowledge Graph Completion: Taxonomy, Progress, and Prospects](https://mlsanity.com/paper/2308.02457/) (2023)
- [DyGMamba: Efficiently Modeling Long-Term Temporal Dependency on Continuous-Time Dynamic Graphs with State Space Models](https://mlsanity.com/paper/2408.04713/) (2024)
- [Graph ODEs and Beyond: A Comprehensive Survey on Integrating Differential Equations with Graph Neural Networks](https://mlsanity.com/paper/2503.23167/) (2025)
- [TFLEX: Temporal Feature-Logic Embedding Framework for Complex Reasoning over Temporal Knowledge Graph](https://mlsanity.com/paper/2205.14307/) (2022)
- [Graph Sequential Neural ODE Process for Link Prediction on Dynamic and Sparse Graphs](https://mlsanity.com/paper/2211.08568/) (2022)
- [zrLLM: Zero-Shot Relational Learning on Temporal Knowledge Graphs with Large Language Models](https://mlsanity.com/paper/2311.10112/) (2023)
- [HiSMatch: Historical Structure Matching based Temporal Knowledge Graph Reasoning](https://mlsanity.com/paper/2210.09708/) (2022)
- [Temporal Fact Reasoning over Hyper-Relational Knowledge Graphs](https://mlsanity.com/paper/2307.10219/) (2023)
- [Chain of History: Learning and Forecasting with LLMs for Temporal Knowledge Graph Completion](https://mlsanity.com/paper/2401.06072/) (2024)
- [ForecastTKGQuestions: A Benchmark for Temporal Question Answering and Forecasting over Temporal Knowledge Graphs](https://mlsanity.com/paper/2208.06501/) (2022)
- [Adaptive Path-Memory Network for Temporal Knowledge Graph Reasoning](https://mlsanity.com/paper/2304.12604/) (2023)
- [TransformerG2G: Adaptive time-stepping for learning temporal graph embeddings using transformers](https://mlsanity.com/paper/2307.02588/) (2023)
- [Temporal Inductive Path Neural Network for Temporal Knowledge Graph Reasoning](https://mlsanity.com/paper/2309.03251/) (2023)
- [A Unified Temporal Knowledge Graph Reasoning Model Towards Interpolation and Extrapolation](https://mlsanity.com/paper/2405.18106/) (2024)
- [Few-Shot Inductive Learning on Temporal Knowledge Graphs using Concept-Aware Information](https://mlsanity.com/paper/2211.08169/) (2022)
- [Local-Global History-aware Contrastive Learning for Temporal Knowledge Graph Reasoning](https://mlsanity.com/paper/2312.01601/) (2023)
- [ECOLA: Enhanced Temporal Knowledge Embeddings with Contextualized Language Representations](https://mlsanity.com/paper/2203.09590/) (2022)
- [Learning Meta Representations of One-shot Relations for Temporal Knowledge Graph Link Prediction](https://mlsanity.com/paper/2205.10621/) (2022)

## Abstract (arXiv)

There has been an increasing interest in inferring future links on temporal knowledge graphs (KG). While links on temporal KGs vary continuously over time, the existing approaches model the temporal KGs in discrete state spaces. To this end, we propose a novel continuum model by extending the idea of neural ordinary differential equations (ODEs) to multi-relational graph convolutional networks. The proposed model preserves the continuous nature of dynamic multi-relational graph data and encodes both temporal and structural information into continuous-time dynamic embeddings. In addition, a novel graph transition layer is applied to capture the transitions on the dynamic graph, i.e., edge formation and dissolution. We perform extensive experiments on five benchmark datasets for temporal KG reasoning, showing our model's superior performance on the future link forecasting task.