# History Repeats: Overcoming Catastrophic Forgetting For Event-Centric Temporal Knowledge Graph Completion

arXiv: https://arxiv.org/abs/2305.18675 · HTML: https://mlsanity.com/paper/2305.18675/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 49 references that are in the map.
- First posted on arXiv 30 May 2023.

## Builds on (most cited first, 29 of 29)

- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Learning without Forgetting](https://mlsanity.com/paper/1606.09282/) (2016)
- [Continual Learning with Deep Generative Replay](https://mlsanity.com/paper/1705.08690/) (2017)
- [Prioritized Experience Replay](https://mlsanity.com/paper/1511.05952/) (2015)
- [Experience Replay for Continual Learning](https://mlsanity.com/paper/1811.11682/) (2018)
- [Know-Evolve: Deep Temporal Reasoning for Dynamic Knowledge Graphs](https://mlsanity.com/paper/1705.05742/) (2017)
- [Learning Sequence Encoders for Temporal Knowledge Graph Completion](https://mlsanity.com/paper/1809.03202/) (2018)
- [Memory Efficient Experience Replay for Streaming Learning](https://mlsanity.com/paper/1809.05922/) (2018)
- [Diachronic Embedding for Temporal Knowledge Graph Completion](https://mlsanity.com/paper/1907.03143/) (2019)
- [SimKGC: Simple Contrastive Knowledge Graph Completion with Pre-trained Language Models](https://mlsanity.com/paper/2203.02167/) (2022)
- [Learning from History: Modeling Temporal Knowledge Graphs with Sequential Copy-Generation Networks](https://mlsanity.com/paper/2012.08492/) (2020)
- [Streaming Graph Neural Networks via Continual Learning](https://mlsanity.com/paper/2009.10951/) (2020)
- [TeMP: Temporal Message Passing for Temporal Knowledge Graph Completion](https://mlsanity.com/paper/2010.03526/) (2020)
- [Continual Learning of Knowledge Graph Embeddings](https://mlsanity.com/paper/2101.05850/) (2021)
- [ChronoR: Rotation Based Temporal Knowledge Graph Embedding](https://mlsanity.com/paper/2103.10379/) (2021)
- [Temporal Knowledge Graph Completion using Box Embeddings](https://mlsanity.com/paper/2109.08970/) (2021)
- [Temporal Knowledge Base Completion: New Algorithms and Evaluation Protocols](https://mlsanity.com/paper/2005.05035/) (2020)
- [DyERNIE: Dynamic Evolution of Riemannian Manifold Embeddings for Temporal Knowledge Graph Completion](https://mlsanity.com/paper/2011.03984/) (2020)
- [Generative Continual Concept Learning](https://mlsanity.com/paper/1906.03744/) (2019)
- [Temporal Knowledge Graph Embedding Model based on Additive Time Series Decomposition](https://mlsanity.com/paper/1911.07893/) (2019)
- [Graph-Based Continual Learning](https://mlsanity.com/paper/2007.04813/) (2020)
- [Multilingual Knowledge Graph Completion with Self-Supervised Adaptive Graph Alignment](https://mlsanity.com/paper/2203.14987/) (2022)
- [Overcoming Concept Shift in Domain-Aware Settings through Consolidated Internal Distributions](https://mlsanity.com/paper/2007.00197/) (2020)
- [Class-Incremental Experience Replay for Continual Learning under Concept Drift](https://mlsanity.com/paper/2104.11861/) (2021)
- [Cognitively Inspired Learning of Incremental Drifting Concepts](https://mlsanity.com/paper/2110.04662/) (2021)
- [One-shot Learning for Temporal Knowledge Graphs](https://mlsanity.com/paper/2010.12144/) (2020)
- [xERTE: Explainable Reasoning on Temporal Knowledge Graphs for Forecasting Future Links](https://mlsanity.com/paper/2012.15537/) (2020)
- [TIE: A Framework for Embedding-based Incremental Temporal Knowledge Graph Completion](https://mlsanity.com/paper/2104.08419/) (2021)
- [Temporal Attribute Prediction via Joint Modeling of Multi-Relational Structure Evolution](https://mlsanity.com/paper/2003.03919/) (2020)

## Built on by (most cited first, 5 of 5)

- [Continual Learning on Graphs: Challenges, Solutions, and Opportunities](https://mlsanity.com/paper/2402.11565/) (2024)
- [Dynamic Transformer Architecture for Continual Learning of Multimodal Tasks](https://mlsanity.com/paper/2401.15275/) (2024)
- [Continuous Unsupervised Domain Adaptation Using Stabilized Representations and Experience Replay](https://mlsanity.com/paper/2402.00580/) (2024)
- [Robust Internal Representations for Domain Generalization](https://mlsanity.com/paper/2309.15522/) (2023)
- [Temporal Knowledge Graph Forecasting under Distribution Shifts: A Synthetic Evaluation](https://mlsanity.com/paper/2607.09232/) (2026)

## Abstract (arXiv)

Temporal knowledge graph (TKG) completion models typically rely on having access to the entire graph during training. However, in real-world scenarios, TKG data is often received incrementally as events unfold, leading to a dynamic non-stationary data distribution over time. While one could incorporate fine-tuning to existing methods to allow them to adapt to evolving TKG data, this can lead to forgetting previously learned patterns. Alternatively, retraining the model with the entire updated TKG can mitigate forgetting but is computationally burdensome. To address these challenges, we propose a general continual training framework that is applicable to any TKG completion method, and leverages two key ideas: (i) a temporal regularization that encourages repurposing of less important model parameters for learning new knowledge, and (ii) a clustering-based experience replay that reinforces the past knowledge by selectively preserving only a small portion of the past data. Our experimental results on widely used event-centric TKG datasets demonstrate the effectiveness of our proposed continual training framework in adapting to new events while reducing catastrophic forgetting. Further, we perform ablation studies to show the effectiveness of each component of our proposed framework. Finally, we investigate the relation between the memory dedicated to experience replay and the benefit gained from our clustering-based sampling strategy.