# Fast and Continual Knowledge Graph Embedding via Incremental LoRA

arXiv: https://arxiv.org/abs/2407.05705 · HTML: https://mlsanity.com/paper/2407.05705/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 41 references that are in the map.
- First posted on arXiv 8 Jul 2024.

## Builds on (most cited first, 22 of 22)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Gradient Episodic Memory for Continual Learning](https://mlsanity.com/paper/1706.08840/) (2017)
- [Progressive Neural Networks](https://mlsanity.com/paper/1606.04671/) (2016)
- [Complex Embeddings for Simple Link Prediction](https://mlsanity.com/paper/1606.06357/) (2016)
- [RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space](https://mlsanity.com/paper/1902.10197/) (2019)
- [Convolutional 2D Knowledge Graph Embeddings](https://mlsanity.com/paper/1707.01476/) (2017)
- [Orthogonal Subspace Learning for Language Model Continual Learning](https://mlsanity.com/paper/2310.14152/) (2023)
- [CORe50: a New Dataset and Benchmark for Continuous Object Recognition](https://mlsanity.com/paper/1705.03550/) (2017)
- [SimplE Embedding for Link Prediction in Knowledge Graphs](https://mlsanity.com/paper/1802.04868/) (2018)
- [Knowledge Transfer for Out-of-Knowledge-Base Entities: A Graph Neural Network Approach](https://mlsanity.com/paper/1706.05674/) (2017)
- [Sentence Embedding Alignment for Lifelong Relation Extraction](https://mlsanity.com/paper/1903.02588/) (2019)
- [Continual Learning of Knowledge Graph Embeddings](https://mlsanity.com/paper/2101.05850/) (2021)
- [Lifelong Embedding Learning and Transfer for Growing Knowledge Graphs](https://mlsanity.com/paper/2211.15845/) (2022)
- [Open Question Answering with Weakly Supervised Embedding Models](https://mlsanity.com/paper/1404.4326/) (2014)
- [Disentangle-based Continual Graph Representation Learning](https://mlsanity.com/paper/2010.02565/) (2020)
- [Towards Continual Knowledge Graph Embedding via Incremental Distillation](https://mlsanity.com/paper/2405.04453/) (2024)
- [FastRE: Towards Fast Relation Extraction with Convolutional Encoder and Improved Cascade Binary Tagging Framework](https://mlsanity.com/paper/2205.02490/) (2022)
- [Entity-Agnostic Representation Learning for Parameter-Efficient Knowledge Graph Embedding](https://mlsanity.com/paper/2302.01849/) (2023)
- [Unlocking Instructive In-Context Learning with Tabular Prompting for Relational Triple Extraction](https://mlsanity.com/paper/2402.13741/) (2024)

## Built on by (most cited first, 14 of 14)

- [Lifelong Learning of Large Language Model based Agents: A Roadmap](https://mlsanity.com/paper/2501.07278/) (2025)
- [Low-Rank Adaptation for Foundation Models: A Comprehensive Review](https://mlsanity.com/paper/2501.00365/) (2024)
- [IGU-LoRA: Adaptive Rank Allocation via Integrated Gradients and Uncertainty-Aware Scoring](https://mlsanity.com/paper/2603.13792/) (2026)
- [Neurosymbolic Methods for Dynamic Knowledge Graphs](https://mlsanity.com/paper/2409.04572/) (2024)
- [Learning to Evolve: Bayesian-Guided Continual Knowledge Graph Embedding](https://mlsanity.com/paper/2508.02426/) (2025)
- [When Modalities Remember: Continual Learning for Multimodal Knowledge Graphs](https://mlsanity.com/paper/2604.02778/) (2026)
- [KGMark: A Diffusion Watermark for Knowledge Graphs](https://mlsanity.com/paper/2505.23873/) (2025)
- [ETT-CKGE: Efficient Task-driven Tokens for Continual Knowledge Graph Embedding](https://mlsanity.com/paper/2506.08158/) (2025)
- [SAGE: Scale-Aware Gradual Evolution for Continual Knowledge Graph Embedding](https://mlsanity.com/paper/2508.11347/) (2025)
- [Improving Continual Learning of Knowledge Graph Embeddings via Informed Initialization](https://mlsanity.com/paper/2511.11118/) (2025)
- [Lightweight Inference-Time Personalization for Frozen Knowledge Graph Embeddings](https://mlsanity.com/paper/2512.22398/) (2025)
- [Revisiting Catastrophic Forgetting in Continual Knowledge Graph Embedding](https://mlsanity.com/paper/2604.19401/) (2026)
- [PrimeKG-CL: A Continual Graph Learning Benchmark on Evolving Biomedical Knowledge Graphs](https://mlsanity.com/paper/2605.10529/) (2026)
- [Matched Excess-Outranker Regularization for Candidate-Set Interference in Continual Knowledge Graph Embedding](https://mlsanity.com/paper/2608.24273/) (2026)

## Abstract (arXiv)

Continual Knowledge Graph Embedding (CKGE) aims to efficiently learn new knowledge and simultaneously preserve old knowledge. Dominant approaches primarily focus on alleviating catastrophic forgetting of old knowledge but neglect efficient learning for the emergence of new knowledge. However, in real-world scenarios, knowledge graphs (KGs) are continuously growing, which brings a significant challenge to fine-tuning KGE models efficiently. To address this issue, we propose a fast CKGE framework (model), incorporating an incremental low-rank adapter (mec) mechanism to efficiently acquire new knowledge while preserving old knowledge. Specifically, to mitigate catastrophic forgetting, model isolates and allocates new knowledge to specific layers based on the fine-grained influence between old and new KGs. Subsequently, to accelerate fine-tuning, model devises an efficient mec mechanism, which embeds the specific layers into incremental low-rank adapters with fewer training parameters. Moreover, mec introduces adaptive rank allocation, which makes the LoRA aware of the importance of entities and adjusts its rank scale adaptively. We conduct experiments on four public datasets and two new datasets with a larger initial scale. Experimental results demonstrate that model can reduce training time by 34%-49% while still achieving competitive link prediction performance against state-of-the-art models on four public datasets (average MRR score of 21.0% vs. 21.1%).Meanwhile, on two newly constructed datasets, model saves 51%-68% training time and improves link prediction performance by 1.5%.