# Lifelong Embedding Learning and Transfer for Growing Knowledge Graphs

arXiv: https://arxiv.org/abs/2211.15845 · HTML: https://mlsanity.com/paper/2211.15845/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 32 references that are in the map.
- First posted on arXiv 29 Nov 2022.

## Builds on (most cited first, 23 of 23)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Gradient Episodic Memory for Continual Learning](https://mlsanity.com/paper/1706.08840/) (2017)
- [Modeling Relational Data with Graph Convolutional Networks](https://mlsanity.com/paper/1703.06103/) (2017)
- [Progressive Neural Networks](https://mlsanity.com/paper/1606.04671/) (2016)
- [Convolutional 2D Knowledge Graph Embeddings](https://mlsanity.com/paper/1707.01476/) (2017)
- [A Survey on Knowledge Graphs: Representation, Acquisition and Applications](https://mlsanity.com/paper/2002.00388/) (2020)
- [GraphMAE: Self-Supervised Masked Graph Autoencoders](https://mlsanity.com/paper/2205.10803/) (2022)
- [Composition-based Multi-Relational Graph Convolutional Networks](https://mlsanity.com/paper/1911.03082/) (2019)
- [CORe50: a New Dataset and Benchmark for Continuous Object Recognition](https://mlsanity.com/paper/1705.03550/) (2017)
- [Neural Bellman-Ford Networks: A General Graph Neural Network Framework for Link Prediction](https://mlsanity.com/paper/2106.06935/) (2021)
- [Inductive Relation Prediction by Subgraph Reasoning](https://mlsanity.com/paper/1911.06962/) (2019)
- [Knowledge Transfer for Out-of-Knowledge-Base Entities: A Graph Neural Network Approach](https://mlsanity.com/paper/1706.05674/) (2017)
- [Learning to Exploit Long-term Relational Dependencies in Knowledge Graphs](https://mlsanity.com/paper/1905.04914/) (2019)
- [NodePiece: Compositional and Parameter-Efficient Representations of Large Knowledge Graphs](https://mlsanity.com/paper/2106.12144/) (2021)
- [Sentence Embedding Alignment for Lifelong Relation Extraction](https://mlsanity.com/paper/1903.02588/) (2019)
- [InteractE: Improving Convolution-based Knowledge Graph Embeddings by Increasing Feature Interactions](https://mlsanity.com/paper/1911.00219/) (2019)
- [Logic Attention Based Neighborhood Aggregation for Inductive Knowledge Graph Embedding](https://mlsanity.com/paper/1811.01399/) (2018)
- [Topology-Aware Correlations Between Relations for Inductive Link Prediction in Knowledge Graphs](https://mlsanity.com/paper/2103.03642/) (2021)
- [Continual Learning of Knowledge Graph Embeddings](https://mlsanity.com/paper/2101.05850/) (2021)
- [Disentangle-based Continual Graph Representation Learning](https://mlsanity.com/paper/2010.02565/) (2020)
- [Efficiently Embedding Dynamic Knowledge Graphs](https://mlsanity.com/paper/1910.06708/) (2019)

## Built on by (most cited first, 28 of 28)

- [Lifelong Learning of Large Language Model based Agents: A Roadmap](https://mlsanity.com/paper/2501.07278/) (2025)
- [Continual Learning on Graphs: Challenges, Solutions, and Opportunities](https://mlsanity.com/paper/2402.11565/) (2024)
- [Continual Graph Learning: A Survey](https://mlsanity.com/paper/2301.12230/) (2023)
- [Graph Learning under Distribution Shifts: A Comprehensive Survey on Domain Adaptation, Out-of-distribution, and Continual Learning](https://mlsanity.com/paper/2402.16374/) (2024)
- [Towards Continual Knowledge Graph Embedding via Incremental Distillation](https://mlsanity.com/paper/2405.04453/) (2024)
- [Generalizing to Unseen Elements: A Survey on Knowledge Extrapolation for Knowledge Graphs](https://mlsanity.com/paper/2302.01859/) (2023)
- [Topology-aware Embedding Memory for Continual Learning on Expanding Networks](https://mlsanity.com/paper/2401.13200/) (2024)
- [Fast and Continual Knowledge Graph Embedding via Incremental LoRA](https://mlsanity.com/paper/2407.05705/) (2024)
- [KnowLA: Enhancing Parameter-efficient Finetuning with Knowledgeable Adaptation](https://mlsanity.com/paper/2403.14950/) (2024)
- [Interactive Continual Learning Architecture for Long-Term Personalization of Home Service Robots](https://mlsanity.com/paper/2403.03462/) (2024)
- [FamilyTool: A Multi-hop Personalized Tool Use Benchmark](https://mlsanity.com/paper/2504.06766/) (2025)
- [Neurosymbolic Methods for Dynamic Knowledge Graphs](https://mlsanity.com/paper/2409.04572/) (2024)
- [Learning to Evolve: Bayesian-Guided Continual Knowledge Graph Embedding](https://mlsanity.com/paper/2508.02426/) (2025)
- [When Modalities Remember: Continual Learning for Multimodal Knowledge Graphs](https://mlsanity.com/paper/2604.02778/) (2026)
- [ETT-CKGE: Efficient Task-driven Tokens for Continual Knowledge Graph Embedding](https://mlsanity.com/paper/2506.08158/) (2025)
- [Dynamic Mixture-of-Experts for Incremental Graph Learning](https://mlsanity.com/paper/2508.09974/) (2025)
- [The Unreasonable Effectiveness of Randomized Representations in Online Continual Graph Learning](https://mlsanity.com/paper/2510.06819/) (2025)
- [Capturing and Anticipating User Intents in Data Analytics via Knowledge Graphs](https://mlsanity.com/paper/2411.01023/) (2024)
- [Towards Improving Long-Tail Entity Predictions in Temporal Knowledge Graphs through Global Similarity and Weighted Sampling](https://mlsanity.com/paper/2507.18977/) (2025)
- [Unlearning of Knowledge Graph Embedding via Preference Optimization](https://mlsanity.com/paper/2507.20566/) (2025)
- [SAGE: Scale-Aware Gradual Evolution for Continual Knowledge Graph Embedding](https://mlsanity.com/paper/2508.11347/) (2025)
- [Improving Continual Learning of Knowledge Graph Embeddings via Informed Initialization](https://mlsanity.com/paper/2511.11118/) (2025)
- [Revisiting Catastrophic Forgetting in Continual Knowledge Graph Embedding](https://mlsanity.com/paper/2604.19401/) (2026)
- [CMKL: Modality-Aware Continual Learning for Evolving Biomedical Knowledge Graphs](https://mlsanity.com/paper/2605.10510/) (2026)
- [PrimeKG-CL: A Continual Graph Learning Benchmark on Evolving Biomedical Knowledge Graphs](https://mlsanity.com/paper/2605.10529/) (2026)
- [MA-DAR: Manifold-Aligned Dynamic Adaptive Routing for Continual Temporal Knowledge Graph Reasoning](https://mlsanity.com/paper/2607.21949/) (2026)
- [Matched Excess-Outranker Regularization for Candidate-Set Interference in Continual Knowledge Graph Embedding](https://mlsanity.com/paper/2608.24273/) (2026)
- [When Successful Knowledge Graph Edits Displace Correct Answers: Rank-Level Locality beyond Parameter Support](https://mlsanity.com/paper/2609.12116/) (2026)

## Abstract (arXiv)

Existing knowledge graph (KG) embedding models have primarily focused on static KGs. However, real-world KGs do not remain static, but rather evolve and grow in tandem with the development of KG applications. Consequently, new facts and previously unseen entities and relations continually emerge, necessitating an embedding model that can quickly learn and transfer new knowledge through growth. Motivated by this, we delve into an expanding field of KG embedding in this paper, i.e., lifelong KG embedding. We consider knowledge transfer and retention of the learning on growing snapshots of a KG without having to learn embeddings from scratch. The proposed model includes a masked KG autoencoder for embedding learning and update, with an embedding transfer strategy to inject the learned knowledge into the new entity and relation embeddings, and an embedding regularization method to avoid catastrophic forgetting. To investigate the impacts of different aspects of KG growth, we construct four datasets to evaluate the performance of lifelong KG embedding. Experimental results show that the proposed model outperforms the state-of-the-art inductive and lifelong embedding baselines.