# ECOLA: Enhanced Temporal Knowledge Embeddings with Contextualized Language Representations

arXiv: https://arxiv.org/abs/2203.09590 · HTML: https://mlsanity.com/paper/2203.09590/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 40 references that are in the map.
- First posted on arXiv 17 Mar 2022.

## Builds on (most cited first, 23 of 23)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation](https://mlsanity.com/paper/1609.08144/) (2016)
- [Embedding Entities and Relations for Learning and Inference in Knowledge Bases](https://mlsanity.com/paper/1412.6575/) (2014)
- [Convolutional 2D Knowledge Graph Embeddings](https://mlsanity.com/paper/1707.01476/) (2017)
- [ERNIE: Enhanced Language Representation with Informative Entities](https://mlsanity.com/paper/1905.07129/) (2019)
- [KEPLER: A Unified Model for Knowledge Embedding and Pre-trained Language Representation](https://mlsanity.com/paper/1911.06136/) (2019)
- [KG-BERT: BERT for Knowledge Graph Completion](https://mlsanity.com/paper/1909.03193/) (2019)
- [K-BERT: Enabling Language Representation with Knowledge Graph](https://mlsanity.com/paper/1909.07606/) (2019)
- [Knowledge Enhanced Contextual Word Representations](https://mlsanity.com/paper/1909.04164/) (2019)
- [SimplE Embedding for Link Prediction in Knowledge Graphs](https://mlsanity.com/paper/1802.04868/) (2018)
- [Tensor Decompositions for temporal knowledge base completion](https://mlsanity.com/paper/2004.04926/) (2020)
- [Diachronic Embedding for Temporal Knowledge Graph Completion](https://mlsanity.com/paper/1907.03143/) (2019)
- [Question Answering Over Temporal Knowledge Graphs](https://mlsanity.com/paper/2106.01515/) (2021)
- [CoLAKE: Contextualized Language and Knowledge Embedding](https://mlsanity.com/paper/2010.00309/) (2020)
- [TimeTraveler: Reinforcement Learning for Temporal Knowledge Graph Forecasting](https://mlsanity.com/paper/2109.04101/) (2021)
- [Temporal Knowledge Graph Forecasting with Neural ODE](https://mlsanity.com/paper/2101.05151/) (2021)
- [TeRo: A Time-aware Knowledge Graph Embedding via Temporal Rotation](https://mlsanity.com/paper/2010.01029/) (2020)
- [DyERNIE: Dynamic Evolution of Riemannian Manifold Embeddings for Temporal Knowledge Graph Completion](https://mlsanity.com/paper/2011.03984/) (2020)
- [Integrating Graph Contextualized Knowledge into Pre-trained Language Models](https://mlsanity.com/paper/1912.00147/) (2019)
- [Embedding Models for Episodic Knowledge Graphs](https://mlsanity.com/paper/1807.00228/) (2018)
- [Temporal Knowledge Graph Embedding Model based on Additive Time Series Decomposition](https://mlsanity.com/paper/1911.07893/) (2019)
- [Learning with Memory Embeddings](https://mlsanity.com/paper/1511.07972/) (2015)

## Built on by (most cited first, 7 of 7)

- [A Survey on Temporal Knowledge Graph: Representation Learning and Applications](https://mlsanity.com/paper/2403.04782/) (2024)
- [zrLLM: Zero-Shot Relational Learning on Temporal Knowledge Graphs with Large Language Models](https://mlsanity.com/paper/2311.10112/) (2023)
- [Graph-O1 : Monte Carlo Tree Search with Reinforcement Learning for Text-Attributed Graph Reasoning](https://mlsanity.com/paper/2512.17912/) (2025)
- [QuAnTS: Question Answering on Time Series](https://mlsanity.com/paper/2511.05124/) (2025)
- [Towards Foundation Model on Temporal Knowledge Graph Reasoning](https://mlsanity.com/paper/2506.06367/) (2025)
- [Multi-hop Reasoning and Retrieval in Embedding Space: Leveraging Large Language Models with Knowledge](https://mlsanity.com/paper/2603.13266/) (2026)
- [Deja vu: Contrastive Historical Modeling with Prefix-tuning for Temporal Knowledge Graph Reasoning](https://mlsanity.com/paper/2404.00051/) (2024)

## Abstract (arXiv)

Since conventional knowledge embedding models cannot take full advantage of the abundant textual information, there have been extensive research efforts in enhancing knowledge embedding using texts. However, existing enhancement approaches cannot apply to temporal knowledge graphs (tKGs), which contain time-dependent event knowledge with complex temporal dynamics. Specifically, existing enhancement approaches often assume knowledge embedding is time-independent. In contrast, the entity embedding in tKG models usually evolves, which poses the challenge of aligning temporally relevant texts with entities. To this end, we propose to study enhancing temporal knowledge embedding with textual data in this paper. As an approach to this task, we propose Enhanced Temporal Knowledge Embeddings with Contextualized Language Representations (ECOLA), which takes the temporal aspect into account and injects textual information into temporal knowledge embedding. To evaluate ECOLA, we introduce three new datasets for training and evaluating ECOLA. Extensive experiments show that ECOLA significantly enhances temporal KG embedding models with up to 287% relative improvements regarding Hits@1 on the link prediction task. The code and models are publicly available on https://anonymous.4open.science/r/ECOLA.