# Pre-trained Language Model with Prompts for Temporal Knowledge Graph Completion

arXiv: https://arxiv.org/abs/2305.07912 · HTML: https://mlsanity.com/paper/2305.07912/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 36 references that are in the map.
- First posted on arXiv 13 May 2023.

## Builds on (most cited first, 24 of 24)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [Language Models as Knowledge Bases?](https://mlsanity.com/paper/1909.01066/) (2019)
- [Modeling Relational Data with Graph Convolutional Networks](https://mlsanity.com/paper/1703.06103/) (2017)
- [Embedding Entities and Relations for Learning and Inference in Knowledge Bases](https://mlsanity.com/paper/1412.6575/) (2014)
- [Complex Embeddings for Simple Link Prediction](https://mlsanity.com/paper/1606.06357/) (2016)
- [RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space](https://mlsanity.com/paper/1902.10197/) (2019)
- [Convolutional 2D Knowledge Graph Embeddings](https://mlsanity.com/paper/1707.01476/) (2017)
- [GPT Understands, Too](https://mlsanity.com/paper/2103.10385/) (2021)
- [KG-BERT: BERT for Knowledge Graph Completion](https://mlsanity.com/paper/1909.03193/) (2019)
- [Factual Probing Is [MASK]: Learning vs. Learning to Recall](https://mlsanity.com/paper/2104.05240/) (2021)
- [Structured Sequence Modeling with Graph Convolutional Recurrent Networks](https://mlsanity.com/paper/1612.07659/) (2016)
- [AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts](https://mlsanity.com/paper/2010.15980/) (2020)
- [PTR: Prompt Tuning with Rules for Text Classification](https://mlsanity.com/paper/2105.11259/) (2021)
- [Learning Sequence Encoders for Temporal Knowledge Graph Completion](https://mlsanity.com/paper/1809.03202/) (2018)
- [End-to-end Structure-Aware Convolutional Networks for Knowledge Base Completion](https://mlsanity.com/paper/1811.04441/) (2018)
- [Tensor Decompositions for temporal knowledge base completion](https://mlsanity.com/paper/2004.04926/) (2020)
- [Temporal Knowledge Graph Reasoning Based on Evolutional Representation Learning](https://mlsanity.com/paper/2104.10353/) (2021)
- [Learning from History: Modeling Temporal Knowledge Graphs with Sequential Copy-Generation Networks](https://mlsanity.com/paper/2012.08492/) (2020)
- [TimeTraveler: Reinforcement Learning for Temporal Knowledge Graph Forecasting](https://mlsanity.com/paper/2109.04101/) (2021)
- [TeMP: Temporal Message Passing for Temporal Knowledge Graph Completion](https://mlsanity.com/paper/2010.03526/) (2020)
- [Tucker decomposition-based Temporal Knowledge Graph Completion](https://mlsanity.com/paper/2011.07751/) (2020)

## Built on by (most cited first, 10 of 10)

- [Large Models for Time Series and Spatio-Temporal Data: A Survey and Outlook](https://mlsanity.com/paper/2310.10196/) (2023)
- [Large Language Models and Knowledge Graphs: Opportunities and Challenges](https://mlsanity.com/paper/2308.06374/) (2023)
- [Large Language Models-guided Dynamic Adaptation for Temporal Knowledge Graph Reasoning](https://mlsanity.com/paper/2405.14170/) (2024)
- [Chain-of-History Reasoning for Temporal Knowledge Graph Forecasting](https://mlsanity.com/paper/2402.14382/) (2024)
- [Rewarding Graph Reasoning Process makes LLMs more Generalized Reasoners](https://mlsanity.com/paper/2503.00845/) (2025)
- [A Comprehensive Evaluation of Large Language Models on Temporal Event Forecasting](https://mlsanity.com/paper/2407.11638/) (2024)
- [Towards Foundation Model on Temporal Knowledge Graph Reasoning](https://mlsanity.com/paper/2506.06367/) (2025)
- [Inductive Reasoning for Temporal Knowledge Graphs with Emerging Entities](https://mlsanity.com/paper/2604.10164/) (2026)
- [Bridging Graph Structure and Knowledge-Guided Editing for Interpretable Temporal Knowledge Graph Reasoning](https://mlsanity.com/paper/2601.21978/) (2026)
- [AdaTKG: Adaptive Memory for Temporal Knowledge Graph Reasoning](https://mlsanity.com/paper/2605.07121/) (2026)

## Abstract (arXiv)

Temporal Knowledge graph completion (TKGC) is a crucial task that involves reasoning at known timestamps to complete the missing part of facts and has attracted more and more attention in recent years. Most existing methods focus on learning representations based on graph neural networks while inaccurately extracting information from timestamps and insufficiently utilizing the implied information in relations. To address these problems, we propose a novel TKGC model, namely Pre-trained Language Model with Prompts for TKGC (PPT). We convert a series of sampled quadruples into pre-trained language model inputs and convert intervals between timestamps into different prompts to make coherent sentences with implicit semantic information. We train our model with a masking strategy to convert TKGC task into a masked token prediction task, which can leverage the semantic information in pre-trained language models. Experiments on three benchmark datasets and extensive analysis demonstrate that our model has great competitiveness compared to other models with four metrics. Our model can effectively incorporate information from temporal knowledge graphs into the language models.