# Continual Learning on Graphs: Challenges, Solutions, and Opportunities

arXiv: https://arxiv.org/abs/2402.11565 · HTML: https://mlsanity.com/paper/2402.11565/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 75 of its 122 references that are in the map.
- First posted on arXiv 18 Feb 2024.

## Builds on (most cited first, 30 of 75)

- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Neural Message Passing for Quantum Chemistry](https://mlsanity.com/paper/1704.01212/) (2017)
- [Learning without Forgetting](https://mlsanity.com/paper/1606.09282/) (2016)
- [iCaRL: Incremental Classifier and Representation Learning](https://mlsanity.com/paper/1611.07725/) (2016)
- [Open Graph Benchmark: Datasets for Machine Learning on Graphs](https://mlsanity.com/paper/2005.00687/) (2020)
- [Gradient Episodic Memory for Continual Learning](https://mlsanity.com/paper/1706.08840/) (2017)
- [Progressive Neural Networks](https://mlsanity.com/paper/1606.04671/) (2016)
- [Continual Lifelong Learning with Neural Networks: A Review](https://mlsanity.com/paper/1802.07569/) (2018)
- [Continual Learning with Deep Generative Replay](https://mlsanity.com/paper/1705.08690/) (2017)
- [Memory Aware Synapses: Learning what (not) to forget](https://mlsanity.com/paper/1711.09601/) (2017)
- [MoleculeNet: A Benchmark for Molecular Machine Learning](https://mlsanity.com/paper/1703.00564/) (2017)
- [Pitfalls of Graph Neural Network Evaluation](https://mlsanity.com/paper/1811.05868/) (2018)
- [Dark Experience for General Continual Learning: a Strong, Simple Baseline](https://mlsanity.com/paper/2004.07211/) (2020)
- [Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigence](https://mlsanity.com/paper/1801.10112/) (2018)
- [Lifelong Learning with Dynamically Expandable Networks](https://mlsanity.com/paper/1708.01547/) (2017)
- [Benchmarking Graph Neural Networks](https://mlsanity.com/paper/2003.00982/) (2020)
- [Three scenarios for continual learning](https://mlsanity.com/paper/1904.07734/) (2019)
- [Understanding over-squashing and bottlenecks on graphs via curvature](https://mlsanity.com/paper/2111.14522/) (2021)
- [Class-incremental learning: survey and performance evaluation on image classification](https://mlsanity.com/paper/2010.15277/) (2020)
- [Provably Powerful Graph Networks](https://mlsanity.com/paper/1905.11136/) (2019)
- [Few-Shot Learning with Graph Neural Networks](https://mlsanity.com/paper/1711.04043/) (2017)
- [A Comprehensive Survey on Graph Anomaly Detection with Deep Learning](https://mlsanity.com/paper/2106.07178/) (2021)
- [Supermasks in Superposition](https://mlsanity.com/paper/2006.14769/) (2020)
- [Meta-Learning Representations for Continual Learning](https://mlsanity.com/paper/1905.12588/) (2019)
- [Self-Supervised Learning of Graph Neural Networks: A Unified Review](https://mlsanity.com/paper/2102.10757/) (2021)
- [Graph-based Anomaly Detection and Description: A Survey](https://mlsanity.com/paper/1404.4679/) (2014)
- [A Comprehensive Survey of Scene Graphs: Generation and Application](https://mlsanity.com/paper/2104.01111/) (2021)

## Built on by (most cited first, 21 of 21)

- [Recent Advances of Multimodal Continual Learning: A Comprehensive Survey](https://mlsanity.com/paper/2410.05352/) (2024)
- [A Survey of Deep Graph Learning under Distribution Shifts: from Graph Out-of-Distribution Generalization to Adaptation](https://mlsanity.com/paper/2410.19265/) (2024)
- [Replay-and-Forget-Free Graph Class-Incremental Learning: A Task Profiling and Prompting Approach](https://mlsanity.com/paper/2410.10341/) (2024)
- [Learning without Isolation: Pathway Protection for Continual Learning](https://mlsanity.com/paper/2505.18568/) (2025)
- [Efficient and Robust Continual Graph Learning for Graph Classification in Biology](https://mlsanity.com/paper/2411.11668/) (2024)
- [Online Continual Graph Learning](https://mlsanity.com/paper/2508.03283/) (2025)
- [Fairness in Augmented Graph Learning: A Survey](https://mlsanity.com/paper/2504.21296/) (2025)
- [Learning System Dynamics without Forgetting](https://mlsanity.com/paper/2407.00717/) (2024)
- [Prompt-Driven Continual Graph Learning](https://mlsanity.com/paper/2502.06327/) (2025)
- [Advances in Continual Graph Learning for Anti-Money Laundering Systems: A Comprehensive Review](https://mlsanity.com/paper/2503.24259/) (2025)
- [HERO: Preserving Structure and Semantics in Heterogeneous Continual Graph Learning](https://mlsanity.com/paper/2505.17458/) (2025)
- [ETT-CKGE: Efficient Task-driven Tokens for Continual Knowledge Graph Embedding](https://mlsanity.com/paper/2506.08158/) (2025)
- [Tackling Distribution Shift in LLM via KILO: Knowledge-Instructed Learning for Continual Adaptation](https://mlsanity.com/paper/2508.03571/) (2025)
- [The Unreasonable Effectiveness of Randomized Representations in Online Continual Graph Learning](https://mlsanity.com/paper/2510.06819/) (2025)
- [AL-GNN: Privacy-Preserving and Replay-Free Continual Graph Learning via Analytic Learning](https://mlsanity.com/paper/2512.18295/) (2025)
- [Empowering Power Outage Prediction with Spatially Aware Hybrid Graph Neural Networks and Contrastive Learning](https://mlsanity.com/paper/2604.04916/) (2026)
- [G2LoRA: Gradient Orthogonal Low-Rank Adaptation Framework for Graph Continual Learning on Text-Attributed Graphs](https://mlsanity.com/paper/2606.01873/) (2026)
- [UFO: A Unified Flow-Oriented Framework for Robust Continual Graph Learning](https://mlsanity.com/paper/2605.09862/) (2026)
- [DRIFT: A Benchmark for Task-Free Continual Graph Learning with Continuous Distribution Shifts](https://mlsanity.com/paper/2605.12998/) (2026)
- [Towards Foundation Models for Relational Databases with Language Models and Graph Neural Networks](https://mlsanity.com/paper/2605.16085/) (2026)
- [HCPN-GCN: Scaling Hierarchical Prototype Networks with Cone Geometry for Continual Graph Learning](https://mlsanity.com/paper/2610.08823/) (2026)

## Abstract (arXiv)

Continual learning on graph data has recently attracted paramount attention for its aim to resolve the catastrophic forgetting problem on existing tasks while adapting the sequentially updated model to newly emerged graph tasks. While there have been efforts to summarize progress on continual learning research over Euclidean data, e.g., images and texts, a systematic review of progress in continual learning on graphs, a.k.a, continual graph learning (CGL) or lifelong graph learning, is still demanding. Graph data are far more complex in terms of data structures and application scenarios, making CGL task settings, model designs, and applications extremely challenging. To bridge the gap, we provide a comprehensive review of existing continual graph learning (CGL) algorithms by elucidating the different task settings and categorizing the existing methods based on their characteristics. We compare the CGL methods with traditional continual learning techniques and analyze the applicability of the traditional continual learning techniques to CGL tasks. Additionally, we review the benchmark works that are crucial to CGL research. Finally, we discuss the remaining challenges and propose several future directions. We will maintain an up-to-date GitHub repository featuring a comprehensive list of CGL algorithms, accessible at https://github.com/UConn-DSIS/Survey-of-Continual-Learning-on-Graphs.