# TimeTraveler: Reinforcement Learning for Temporal Knowledge Graph Forecasting

arXiv: https://arxiv.org/abs/2109.04101 · HTML: https://mlsanity.com/paper/2109.04101/

## Key facts

- Cited by 67 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 40 references that are in the map.
- First posted on arXiv 9 Sep 2021.

## Builds on (most cited first, 24 of 24)

- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [Modeling Relational Data with Graph Convolutional Networks](https://mlsanity.com/paper/1703.06103/) (2017)
- [Complex Embeddings for Simple Link Prediction](https://mlsanity.com/paper/1606.06357/) (2016)
- [RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space](https://mlsanity.com/paper/1902.10197/) (2019)
- [Convolutional 2D Knowledge Graph Embeddings](https://mlsanity.com/paper/1707.01476/) (2017)
- [Knowledge Graphs](https://mlsanity.com/paper/2003.02320/) (2020)
- [DeepPath: A Reinforcement Learning Method for Knowledge Graph Reasoning](https://mlsanity.com/paper/1707.06690/) (2017)
- [Low-Dimensional Hyperbolic Knowledge Graph Embeddings](https://mlsanity.com/paper/2005.00545/) (2020)
- [Variational Reasoning for Question Answering with Knowledge Graph](https://mlsanity.com/paper/1709.04071/) (2017)
- [A Survey on Knowledge Graph-Based Recommender Systems](https://mlsanity.com/paper/2003.00911/) (2020)
- [Go for a Walk and Arrive at the Answer: Reasoning Over Paths in Knowledge Bases using Reinforcement Learning](https://mlsanity.com/paper/1711.05851/) (2017)
- [Know-Evolve: Deep Temporal Reasoning for Dynamic Knowledge Graphs](https://mlsanity.com/paper/1705.05742/) (2017)
- [Learning Sequence Encoders for Temporal Knowledge Graph Completion](https://mlsanity.com/paper/1809.03202/) (2018)
- [Learning Attention-based Embeddings for Relation Prediction in Knowledge Graphs](https://mlsanity.com/paper/1906.01195/) (2019)
- [Tensor Decompositions for temporal knowledge base completion](https://mlsanity.com/paper/2004.04926/) (2020)
- [Diachronic Embedding for Temporal Knowledge Graph Completion](https://mlsanity.com/paper/1907.03143/) (2019)
- [Multi-Hop Knowledge Graph Reasoning with Reward Shaping](https://mlsanity.com/paper/1808.10568/) (2018)
- [Learning from History: Modeling Temporal Knowledge Graphs with Sequential Copy-Generation Networks](https://mlsanity.com/paper/2012.08492/) (2020)
- [Learning to Exploit Long-term Relational Dependencies in Knowledge Graphs](https://mlsanity.com/paper/1905.04914/) (2019)
- [TeMP: Temporal Message Passing for Temporal Knowledge Graph Completion](https://mlsanity.com/paper/2010.03526/) (2020)
- [InteractE: Improving Convolution-based Knowledge Graph Embeddings by Increasing Feature Interactions](https://mlsanity.com/paper/1911.00219/) (2019)
- [Modeling Relation Paths for Representation Learning of Knowledge Bases](https://mlsanity.com/paper/1506.00379/) (2015)
- [DyERNIE: Dynamic Evolution of Riemannian Manifold Embeddings for Temporal Knowledge Graph Completion](https://mlsanity.com/paper/2011.03984/) (2020)
- [T-GAP: Learning to Walk across Time for Temporal Knowledge Graph Completion](https://mlsanity.com/paper/2012.10595/) (2020)

## Built on by (most cited first, 30 of 67)

- [A Survey of Knowledge Graph Reasoning on Graph Types: Static, Dynamic, and Multimodal](https://mlsanity.com/paper/2212.05767/) (2022)
- [Large Language Models-guided Dynamic Adaptation for Temporal Knowledge Graph Reasoning](https://mlsanity.com/paper/2405.14170/) (2024)
- [Temporal Knowledge Graph Forecasting Without Knowledge Using In-Context Learning](https://mlsanity.com/paper/2305.10613/) (2023)
- [TGB 2.0: A Benchmark for Learning on Temporal Knowledge Graphs and Heterogeneous Graphs](https://mlsanity.com/paper/2406.09639/) (2024)
- [Complex Evolutional Pattern Learning for Temporal Knowledge Graph Reasoning](https://mlsanity.com/paper/2203.07782/) (2022)
- [GenTKG: Generative Forecasting on Temporal Knowledge Graph with Large Language Models](https://mlsanity.com/paper/2310.07793/) (2023)
- [A Survey on Temporal Knowledge Graph: Representation Learning and Applications](https://mlsanity.com/paper/2403.04782/) (2024)
- [Reinforcement learning on graphs: A survey](https://mlsanity.com/paper/2204.06127/) (2022)
- [What is Event Knowledge Graph: A Survey](https://mlsanity.com/paper/2112.15280/) (2021)
- [History repeats Itself: A Baseline for Temporal Knowledge Graph Forecasting](https://mlsanity.com/paper/2404.16726/) (2024)
- [A Survey on Temporal Knowledge Graph Completion: Taxonomy, Progress, and Prospects](https://mlsanity.com/paper/2308.02457/) (2023)
- [zrLLM: Zero-Shot Relational Learning on Temporal Knowledge Graphs with Large Language Models](https://mlsanity.com/paper/2311.10112/) (2023)
- [HiSMatch: Historical Structure Matching based Temporal Knowledge Graph Reasoning](https://mlsanity.com/paper/2210.09708/) (2022)
- [Pre-trained Language Model with Prompts for Temporal Knowledge Graph Completion](https://mlsanity.com/paper/2305.07912/) (2023)
- [On the Evolution of Knowledge Graphs: A Survey and Perspective](https://mlsanity.com/paper/2310.04835/) (2023)
- [Chain of History: Learning and Forecasting with LLMs for Temporal Knowledge Graph Completion](https://mlsanity.com/paper/2401.06072/) (2024)
- [Adaptive Path-Memory Network for Temporal Knowledge Graph Reasoning](https://mlsanity.com/paper/2304.12604/) (2023)
- [Temporal Inductive Path Neural Network for Temporal Knowledge Graph Reasoning](https://mlsanity.com/paper/2309.03251/) (2023)
- [A Unified Temporal Knowledge Graph Reasoning Model Towards Interpolation and Extrapolation](https://mlsanity.com/paper/2405.18106/) (2024)
- [Few-Shot Inductive Learning on Temporal Knowledge Graphs using Concept-Aware Information](https://mlsanity.com/paper/2211.08169/) (2022)
- [Local-Global History-aware Contrastive Learning for Temporal Knowledge Graph Reasoning](https://mlsanity.com/paper/2312.01601/) (2023)
- [Transformer-based Reasoning for Learning Evolutionary Chain of Events on Temporal Knowledge Graph](https://mlsanity.com/paper/2405.00352/) (2024)
- [A Comprehensive Evaluation of Large Language Models on Temporal Event Forecasting](https://mlsanity.com/paper/2407.11638/) (2024)
- [ECOLA: Enhanced Temporal Knowledge Embeddings with Contextualized Language Representations](https://mlsanity.com/paper/2203.09590/) (2022)
- [GTRL: An Entity Group-Aware Temporal Knowledge Graph Representation Learning Method](https://mlsanity.com/paper/2302.11091/) (2023)
- [A Primer on Temporal Graph Learning](https://mlsanity.com/paper/2401.03988/) (2024)
- [DREAM: Adaptive Reinforcement Learning based on Attention Mechanism for Temporal Knowledge Graph Reasoning](https://mlsanity.com/paper/2304.03984/) (2023)
- [ONSEP: A Novel Online Neural-Symbolic Framework for Event Prediction Based on Large Language Model](https://mlsanity.com/paper/2408.07840/) (2024)
- [DECRL: A Deep Evolutionary Clustering Jointed Temporal Knowledge Graph Representation Learning Approach](https://mlsanity.com/paper/2410.22631/) (2024)
- [Disentangled Multi-span Evolutionary Network against Temporal Knowledge Graph Reasoning](https://mlsanity.com/paper/2505.14020/) (2025)

## Abstract (arXiv)

Temporal knowledge graph (TKG) reasoning is a crucial task that has gained increasing research interest in recent years. Most existing methods focus on reasoning at past timestamps to complete the missing facts, and there are only a few works of reasoning on known TKGs to forecast future facts. Compared with the completion task, the forecasting task is more difficult that faces two main challenges: (1) how to effectively model the time information to handle future timestamps? (2) how to make inductive inference to handle previously unseen entities that emerge over time? To address these challenges, we propose the first reinforcement learning method for forecasting. Specifically, the agent travels on historical knowledge graph snapshots to search for the answer. Our method defines a relative time encoding function to capture the timespan information, and we design a novel time-shaped reward based on Dirichlet distribution to guide the model learning. Furthermore, we propose a novel representation method for unseen entities to improve the inductive inference ability of the model. We evaluate our method for this link prediction task at future timestamps. Extensive experiments on four benchmark datasets demonstrate substantial performance improvement meanwhile with higher explainability, less calculation, and fewer parameters when compared with existing state-of-the-art methods.