# Continual Learning on Dynamic Graphs via Parameter Isolation

arXiv: https://arxiv.org/abs/2305.13825 · HTML: https://mlsanity.com/paper/2305.13825/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- Builds on 51 of its 68 references that are in the map.
- First posted on arXiv 23 May 2023.

## Builds on (most cited first, 30 of 51)

- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [How Powerful are Graph Neural Networks?](https://mlsanity.com/paper/1810.00826/) (2018)
- [Learning without Forgetting](https://mlsanity.com/paper/1606.09282/) (2016)
- [iCaRL: Incremental Classifier and Representation Learning](https://mlsanity.com/paper/1611.07725/) (2016)
- [node2vec: Scalable Feature Learning for Networks](https://mlsanity.com/paper/1607.00653/) (2016)
- [Gradient Episodic Memory for Continual Learning](https://mlsanity.com/paper/1706.08840/) (2017)
- [DeepWalk: Online Learning of Social Representations](https://mlsanity.com/paper/1403.6652/) (2014)
- [Simplifying Graph Convolutional Networks](https://mlsanity.com/paper/1902.07153/) (2019)
- [Progressive Neural Networks](https://mlsanity.com/paper/1606.04671/) (2016)
- [Continual Learning with Deep Generative Replay](https://mlsanity.com/paper/1705.08690/) (2017)
- [LightGCN: Simplifying and Powering Graph Convolution Network for Recommendation](https://mlsanity.com/paper/2002.02126/) (2020)
- [Memory Aware Synapses: Learning what (not) to forget](https://mlsanity.com/paper/1711.09601/) (2017)
- [Experience Replay for Continual Learning](https://mlsanity.com/paper/1811.11682/) (2018)
- [Pitfalls of Graph Neural Network Evaluation](https://mlsanity.com/paper/1811.05868/) (2018)
- [Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigence](https://mlsanity.com/paper/1801.10112/) (2018)
- [Lifelong Learning with Dynamically Expandable Networks](https://mlsanity.com/paper/1708.01547/) (2017)
- [LINE: Large-scale Information Network Embedding](https://mlsanity.com/paper/1503.03578/) (2015)
- [Directional Message Passing for Molecular Graphs](https://mlsanity.com/paper/2003.03123/) (2020)
- [Predicting Dynamic Embedding Trajectory in Temporal Interaction Networks](https://mlsanity.com/paper/1908.01207/) (2019)
- [EvolveGCN: Evolving Graph Convolutional Networks for Dynamic Graphs](https://mlsanity.com/paper/1902.10191/) (2019)
- [PathNet: Evolution Channels Gradient Descent in Super Neural Networks](https://mlsanity.com/paper/1701.08734/) (2017)
- [Fast and Uncertainty-Aware Directional Message Passing for Non-Equilibrium Molecules](https://mlsanity.com/paper/2011.14115/) (2020)
- [Compacting, Picking and Growing for Unforgetting Continual Learning](https://mlsanity.com/paper/1910.06562/) (2019)
- [Anti-Money Laundering in Bitcoin: Experimenting with Graph Convolutional Networks for Financial Forensics](https://mlsanity.com/paper/1908.02591/) (2019)
- [Learning on Large-scale Text-attributed Graphs via Variational Inference](https://mlsanity.com/paper/2210.14709/) (2022)
- [Overcoming Catastrophic Forgetting in Incremental Few-Shot Learning by Finding Flat Minima](https://mlsanity.com/paper/2111.01549/) (2021)

## Built on by (most cited first, 30 of 36)

- [A Comprehensive Survey of Dynamic Graph Neural Networks: Models, Frameworks, Benchmarks, Experiments and Challenges](https://mlsanity.com/paper/2405.00476/) (2024)
- [Continual Learning on Graphs: Challenges, Solutions, and Opportunities](https://mlsanity.com/paper/2402.11565/) (2024)
- [Graph Learning under Distribution Shifts: A Comprehensive Survey on Domain Adaptation, Out-of-distribution, and Continual Learning](https://mlsanity.com/paper/2402.16374/) (2024)
- [Topology-aware Embedding Memory for Continual Learning on Expanding Networks](https://mlsanity.com/paper/2401.13200/) (2024)
- [Replay-and-Forget-Free Graph Class-Incremental Learning: A Task Profiling and Prompting Approach](https://mlsanity.com/paper/2410.10341/) (2024)
- [A Topology-aware Graph Coarsening Framework for Continual Graph Learning](https://mlsanity.com/paper/2401.03077/) (2024)
- [On Manipulating Signals of User-Item Graph: A Jacobi Polynomial-based Graph Collaborative Filtering](https://mlsanity.com/paper/2306.03624/) (2023)
- [Prototype-Based Continual Learning with Label-free Replay Buffer and Cluster Preservation Loss](https://mlsanity.com/paper/2504.07240/) (2025)
- [GCAL: Adapting Graph Models to Evolving Domain Shifts](https://mlsanity.com/paper/2505.16860/) (2025)
- [Foundation Model-oriented Robustness: Robust Image Model Evaluation with Pretrained Models](https://mlsanity.com/paper/2308.10632/) (2023)
- [Advancing continual lifelong learning in neural information retrieval: definition, dataset, framework, and empirical evaluation](https://mlsanity.com/paper/2308.08378/) (2023)
- [Graph Memory Learning: Imitating Lifelong Remembering and Forgetting of Brain Networks](https://mlsanity.com/paper/2407.19183/) (2024)
- [Incremental Causal Graph Learning for Online Cyberattack Detection in Cyber-Physical Infrastructures](https://mlsanity.com/paper/2507.14387/) (2025)
- [Distribution Shift Aware Neural Tabular Learning](https://mlsanity.com/paper/2508.19486/) (2025)
- [Self-Evolving Agents as Dynamic Graph Transformation: A Survey and New Perspective](https://mlsanity.com/paper/2608.18104/) (2026)
- [Parameter-Efficient Fine-Tuning for Continual Learning: A Neural Tangent Kernel Perspective](https://mlsanity.com/paper/2407.17120/) (2024)
- [Efficient and Robust Continual Graph Learning for Graph Classification in Biology](https://mlsanity.com/paper/2411.11668/) (2024)
- [Adapter-Enhanced Semantic Prompting for Continual Learning](https://mlsanity.com/paper/2412.11074/) (2024)
- [Question-Adaptive Graph Learning for Multi-hop Retrieval Augmented Generation](https://mlsanity.com/paper/2510.11541/) (2025)
- [Continual Low-Rank Adapters for LLM-based Generative Recommender Systems](https://mlsanity.com/paper/2510.25093/) (2025)
- [Review of Explainable Graph-Based Recommender Systems](https://mlsanity.com/paper/2408.00166/) (2024)
- [Prompt-Driven Continual Graph Learning](https://mlsanity.com/paper/2502.06327/) (2025)
- [Advances in Continual Graph Learning for Anti-Money Laundering Systems: A Comprehensive Review](https://mlsanity.com/paper/2503.24259/) (2025)
- [Outlook Towards Deployable Continual Learning for Particle Accelerators](https://mlsanity.com/paper/2504.03793/) (2025)
- [HERO: Preserving Structure and Semantics in Heterogeneous Continual Graph Learning](https://mlsanity.com/paper/2505.17458/) (2025)
- [Dynamic Mixture-of-Experts for Incremental Graph Learning](https://mlsanity.com/paper/2508.09974/) (2025)
- [AL-GNN: Privacy-Preserving and Replay-Free Continual Graph Learning via Analytic Learning](https://mlsanity.com/paper/2512.18295/) (2025)
- [TAAM:Inductive Graph-Class Incremental Learning with Task-Aware Adaptive Modulation](https://mlsanity.com/paper/2602.08036/) (2026)
- [G2LoRA: Gradient Orthogonal Low-Rank Adaptation Framework for Graph Continual Learning on Text-Attributed Graphs](https://mlsanity.com/paper/2606.01873/) (2026)
- [MEGA: Second-Order Gradient Alignment for Catastrophic Forgetting Mitigation in GFSCIL](https://mlsanity.com/paper/2504.13691/) (2025)

## Abstract (arXiv)

Many real-world graph learning tasks require handling dynamic graphs where new nodes and edges emerge. Dynamic graph learning methods commonly suffer from the catastrophic forgetting problem, where knowledge learned for previous graphs is overwritten by updates for new graphs. To alleviate the problem, continual graph learning methods are proposed. However, existing continual graph learning methods aim to learn new patterns and maintain old ones with the same set of parameters of fixed size, and thus face a fundamental tradeoff between both goals. In this paper, we propose Parameter Isolation GNN (PI-GNN) for continual learning on dynamic graphs that circumvents the tradeoff via parameter isolation and expansion. Our motivation lies in that different parameters contribute to learning different graph patterns. Based on the idea, we expand model parameters to continually learn emerging graph patterns. Meanwhile, to effectively preserve knowledge for unaffected patterns, we find parameters that correspond to them via optimization and freeze them to prevent them from being rewritten. Experiments on eight real-world datasets corroborate the effectiveness of PI-GNN compared to state-of-the-art baselines.