# SAGE: Scale-Aware Gradual Evolution for Continual Knowledge Graph Embedding

arXiv: https://arxiv.org/abs/2508.11347 · HTML: https://mlsanity.com/paper/2508.11347/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 27 of its 40 references that are in the map.
- First posted on arXiv 15 Aug 2025.

## Builds on (most cited first, 27 of 27)

- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Gradient Episodic Memory for Continual Learning](https://mlsanity.com/paper/1706.08840/) (2017)
- [Progressive Neural Networks](https://mlsanity.com/paper/1606.04671/) (2016)
- [Embedding Entities and Relations for Learning and Inference in Knowledge Bases](https://mlsanity.com/paper/1412.6575/) (2014)
- [Complex Embeddings for Simple Link Prediction](https://mlsanity.com/paper/1606.06357/) (2016)
- [LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attention](https://mlsanity.com/paper/2303.16199/) (2023)
- [Scaling Laws for Autoregressive Generative Modeling](https://mlsanity.com/paper/2010.14701/) (2020)
- [Rigging the Lottery: Making All Tickets Winners](https://mlsanity.com/paper/1911.11134/) (2019)
- [EvolveGCN: Evolving Graph Convolutional Networks for Dynamic Graphs](https://mlsanity.com/paper/1902.10191/) (2019)
- [KGAT: Knowledge Graph Attention Network for Recommendation](https://mlsanity.com/paper/1905.07854/) (2019)
- [CORe50: a New Dataset and Benchmark for Continuous Object Recognition](https://mlsanity.com/paper/1705.03550/) (2017)
- [Representation Learning for Dynamic Graphs: A Survey](https://mlsanity.com/paper/1905.11485/) (2019)
- [SimplE Embedding for Link Prediction in Knowledge Graphs](https://mlsanity.com/paper/1802.04868/) (2018)
- [Knowledge Transfer for Out-of-Knowledge-Base Entities: A Graph Neural Network Approach](https://mlsanity.com/paper/1706.05674/) (2017)
- [Knowledge Graph Embedding for Link Prediction: A Comparative Analysis](https://mlsanity.com/paper/2002.00819/) (2020)
- [Sentence Embedding Alignment for Lifelong Relation Extraction](https://mlsanity.com/paper/1903.02588/) (2019)
- [LoRA ensembles for large language model fine-tuning](https://mlsanity.com/paper/2310.00035/) (2023)
- [Continual Learning of Knowledge Graph Embeddings](https://mlsanity.com/paper/2101.05850/) (2021)
- [EvoChart: A Benchmark and a Self-Training Approach Towards Real-World Chart Understanding](https://mlsanity.com/paper/2409.01577/) (2024)
- [Lifelong Embedding Learning and Transfer for Growing Knowledge Graphs](https://mlsanity.com/paper/2211.15845/) (2022)
- [Disentangle-based Continual Graph Representation Learning](https://mlsanity.com/paper/2010.02565/) (2020)
- [Scaling Law of Large Sequential Recommendation Models](https://mlsanity.com/paper/2311.11351/) (2023)
- [Sparsing Law: Towards Large Language Models with Greater Activation Sparsity](https://mlsanity.com/paper/2411.02335/) (2024)
- [Scaling Laws for Acoustic Models](https://mlsanity.com/paper/2106.09488/) (2021)
- [Towards Continual Knowledge Graph Embedding via Incremental Distillation](https://mlsanity.com/paper/2405.04453/) (2024)
- [Fast and Continual Knowledge Graph Embedding via Incremental LoRA](https://mlsanity.com/paper/2407.05705/) (2024)

## Abstract (arXiv)

Traditional knowledge graph (KG) embedding methods aim to represent entities and relations in a low-dimensional space, primarily focusing on static graphs. However, real-world KGs are dynamically evolving with the constant addition of entities, relations and facts. To address such dynamic nature of KGs, several continual knowledge graph embedding (CKGE) methods have been developed to efficiently update KG embeddings to accommodate new facts while maintaining learned knowledge. As KGs grow at different rates and scales in real-world scenarios, existing CKGE methods often fail to consider the varying scales of updates and lack systematic evaluation throughout the entire update process. In this paper, we propose SAGE, a scale-aware gradual evolution framework for CKGE. Specifically, SAGE firstly determine the embedding dimensions based on the update scales and expand the embedding space accordingly. The Dynamic Distillation mechanism is further employed to balance the preservation of learned knowledge and the incorporation of new facts. We conduct extensive experiments on seven benchmarks, and the results show that SAGE consistently outperforms existing baselines, with a notable improvement of 1.38% in MRR, 1.25% in H@1 and 1.6% in H@10. Furthermore, experiments comparing SAGE with methods using fixed embedding dimensions show that SAGE achieves optimal performance on every snapshot, demonstrating the importance of adaptive embedding dimensions in CKGE. The codes of SAGE are publicly available at: https://github.com/lyfxjtu/Dynamic-Embedding.