# Continual Learning of Knowledge Graph Embeddings

arXiv: https://arxiv.org/abs/2101.05850 · HTML: https://mlsanity.com/paper/2101.05850/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 33 references that are in the map.
- First posted on arXiv 14 Jan 2021.

## Builds on (most cited first, 20 of 20)

- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Progressive Neural Networks](https://mlsanity.com/paper/1606.04671/) (2016)
- [Continual Lifelong Learning with Neural Networks: A Review](https://mlsanity.com/paper/1802.07569/) (2018)
- [Continual Learning with Deep Generative Replay](https://mlsanity.com/paper/1705.08690/) (2017)
- [Convolutional 2D Knowledge Graph Embeddings](https://mlsanity.com/paper/1707.01476/) (2017)
- [Generating Sentences from a Continuous Space](https://mlsanity.com/paper/1511.06349/) (2015)
- [CORe50: a New Dataset and Benchmark for Continuous Object Recognition](https://mlsanity.com/paper/1705.03550/) (2017)
- [A Review of Relational Machine Learning for Knowledge Graphs](https://mlsanity.com/paper/1503.00759/) (2015)
- [Representation Learning for Dynamic Graphs: A Survey](https://mlsanity.com/paper/1905.11485/) (2019)
- [Re-evaluating Continual Learning Scenarios: A Categorization and Case for Strong Baselines](https://mlsanity.com/paper/1810.12488/) (2018)
- [Go for a Walk and Arrive at the Answer: Reasoning Over Paths in Knowledge Bases using Reinforcement Learning](https://mlsanity.com/paper/1711.05851/) (2017)
- [Don't forget, there is more than forgetting: new metrics for Continual Learning](https://mlsanity.com/paper/1810.13166/) (2018)
- [Generative replay with feedback connections as a general strategy for continual learning](https://mlsanity.com/paper/1809.10635/) (2018)
- [Analogical Inference for Multi-Relational Embeddings](https://mlsanity.com/paper/1705.02426/) (2017)
- [KBGAN: Adversarial Learning for Knowledge Graph Embeddings](https://mlsanity.com/paper/1711.04071/) (2017)
- [Incorporating GAN for Negative Sampling in Knowledge Representation Learning](https://mlsanity.com/paper/1809.11017/) (2018)
- [Efficiently Embedding Dynamic Knowledge Graphs](https://mlsanity.com/paper/1910.06708/) (2019)
- [RoboBrain: Large-Scale Knowledge Engine for Robots](https://mlsanity.com/paper/1412.0691/) (2014)
- [RoboCSE: Robot Common Sense Embedding](https://mlsanity.com/paper/1903.00412/) (2019)
- [A Motion Taxonomy for Manipulation Embedding](https://mlsanity.com/paper/2007.06695/) (2020)

## Built on by (most cited first, 30 of 35)

- [InsCL: A Data-efficient Continual Learning Paradigm for Fine-tuning Large Language Models with Instructions](https://mlsanity.com/paper/2403.11435/) (2024)
- [CaT: Balanced Continual Graph Learning with Graph Condensation](https://mlsanity.com/paper/2309.09455/) (2023)
- [Complex Evolutional Pattern Learning for Temporal Knowledge Graph Reasoning](https://mlsanity.com/paper/2203.07782/) (2022)
- [Lifelong Embedding Learning and Transfer for Growing Knowledge Graphs](https://mlsanity.com/paper/2211.15845/) (2022)
- [Continual Learning on Graphs: Challenges, Solutions, and Opportunities](https://mlsanity.com/paper/2402.11565/) (2024)
- [Towards Robust Graph Incremental Learning on Evolving Graphs](https://mlsanity.com/paper/2402.12987/) (2024)
- [ConPET: Continual Parameter-Efficient Tuning for Large Language Models](https://mlsanity.com/paper/2309.14763/) (2023)
- [Towards Continual Knowledge Graph Embedding via Incremental Distillation](https://mlsanity.com/paper/2405.04453/) (2024)
- [Topology-aware Embedding Memory for Continual Learning on Expanding Networks](https://mlsanity.com/paper/2401.13200/) (2024)
- [Fast and Continual Knowledge Graph Embedding via Incremental LoRA](https://mlsanity.com/paper/2407.05705/) (2024)
- [Aligning Robot and Human Representations](https://mlsanity.com/paper/2302.01928/) (2023)
- [PUMA: Efficient Continual Graph Learning for Node Classification with Graph Condensation](https://mlsanity.com/paper/2312.14439/) (2023)
- [Continual Learning for Smart City: A Survey](https://mlsanity.com/paper/2404.00983/) (2024)
- [BeGin: Extensive Benchmark Scenarios and An Easy-to-use Framework for Graph Continual Learning](https://mlsanity.com/paper/2211.14568/) (2022)
- [On the Limitation and Experience Replay for GNNs in Continual Learning](https://mlsanity.com/paper/2302.03534/) (2023)
- [Scalable and Efficient Continual Learning from Demonstration via a Hypernetwork-generated Stable Dynamics Model](https://mlsanity.com/paper/2311.03600/) (2023)
- [Towards Continuous Reuse of Graph Models via Holistic Memory Diversification](https://mlsanity.com/paper/2406.07413/) (2024)
- [Survey on English Entity Linking on Wikidata](https://mlsanity.com/paper/2112.01989/) (2021)
- [Interactive Continual Learning Architecture for Long-Term Personalization of Home Service Robots](https://mlsanity.com/paper/2403.03462/) (2024)
- [History Repeats: Overcoming Catastrophic Forgetting For Event-Centric Temporal Knowledge Graph Completion](https://mlsanity.com/paper/2305.18675/) (2023)
- [NC-NCD: Novel Class Discovery for Node Classification](https://mlsanity.com/paper/2407.17816/) (2024)
- [Neurosymbolic Methods for Dynamic Knowledge Graphs](https://mlsanity.com/paper/2409.04572/) (2024)
- [Wikidated 1.0: An Evolving Knowledge Graph Dataset of Wikidata's Revision History](https://mlsanity.com/paper/2112.05003/) (2021)
- [Online Continual Learning via the Knowledge Invariant and Spread-out Properties](https://mlsanity.com/paper/2302.00858/) (2023)
- [When LLMs meet open-world graph learning: a new perspective for unlabeled data uncertainty](https://mlsanity.com/paper/2505.13989/) (2025)
- [Elastic Weight Consolidation for Knowledge Graph Continual Learning: An Empirical Evaluation](https://mlsanity.com/paper/2512.01890/) (2025)
- [Constraint Multi-class Positive and Unlabeled Learning for Distantly Supervised Named Entity Recognition](https://mlsanity.com/paper/2504.04963/) (2025)
- [Towards Improving Long-Tail Entity Predictions in Temporal Knowledge Graphs through Global Similarity and Weighted Sampling](https://mlsanity.com/paper/2507.18977/) (2025)
- [Unlearning of Knowledge Graph Embedding via Preference Optimization](https://mlsanity.com/paper/2507.20566/) (2025)
- [SAGE: Scale-Aware Gradual Evolution for Continual Knowledge Graph Embedding](https://mlsanity.com/paper/2508.11347/) (2025)

## Abstract (arXiv)

In recent years, there has been a resurgence in methods that use distributed (neural) representations to represent and reason about semantic knowledge for robotics applications. However, while robots often observe previously unknown concepts, these representations typically assume that all concepts are known a priori, and incorporating new information requires all concepts to be learned afresh. Our work relaxes this limiting assumption of existing representations and tackles the incremental knowledge graph embedding problem by leveraging the principles of a range of continual learning methods. Through an experimental evaluation with several knowledge graphs and embedding representations, we provide insights about trade-offs for practitioners to match a semantics-driven robotics applications to a suitable continual knowledge graph embedding method.