# Streaming Graph Neural Networks via Continual Learning

arXiv: https://arxiv.org/abs/2009.10951 · HTML: https://mlsanity.com/paper/2009.10951/

## Key facts

- Cited by 64 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Sep 2020.

## Built on by (most cited first, 30 of 64)

- [A Survey on Graph Representation Learning Methods](https://mlsanity.com/paper/2204.01855/) (2022)
- [TrafficStream: A Streaming Traffic Flow Forecasting Framework Based on Graph Neural Networks and Continual Learning](https://mlsanity.com/paper/2106.06273/) (2021)
- [Online Graph Dictionary Learning](https://mlsanity.com/paper/2102.06555/) (2021)
- [Continual Learning on Dynamic Graphs via Parameter Isolation](https://mlsanity.com/paper/2305.13825/) (2023)
- [Self-Supervised Continual Graph Learning in Adaptive Riemannian Spaces](https://mlsanity.com/paper/2211.17068/) (2022)
- [Graph Lifelong Learning: A Survey](https://mlsanity.com/paper/2202.10688/) (2022)
- [Hierarchical Prototype Networks for Continual Graph Representation Learning](https://mlsanity.com/paper/2111.15422/) (2021)
- [Prompt-augmented Temporal Point Process for Streaming Event Sequence](https://mlsanity.com/paper/2310.04993/) (2023)
- [Continual Learning on Graphs: Challenges, Solutions, and Opportunities](https://mlsanity.com/paper/2402.11565/) (2024)
- [Continual Graph Learning: A Survey](https://mlsanity.com/paper/2301.12230/) (2023)
- [Towards Robust Graph Incremental Learning on Evolving Graphs](https://mlsanity.com/paper/2402.12987/) (2024)
- [Graph Learning under Distribution Shifts: A Comprehensive Survey on Domain Adaptation, Out-of-distribution, and Continual Learning](https://mlsanity.com/paper/2402.16374/) (2024)
- [Causal Incremental Graph Convolution for Recommender System Retraining](https://mlsanity.com/paper/2108.06889/) (2021)
- [Topology-aware Embedding Memory for Continual Learning on Expanding Networks](https://mlsanity.com/paper/2401.13200/) (2024)
- [DSLR: Diversity Enhancement and Structure Learning for Rehearsal-based Graph Continual Learning](https://mlsanity.com/paper/2402.13711/) (2024)
- [Towards Open Temporal Graph Neural Networks](https://mlsanity.com/paper/2303.15015/) (2023)
- [Collaboration-Aware Graph Convolutional Network for Recommender Systems](https://mlsanity.com/paper/2207.06221/) (2022)
- [Continual Learning on Graphs: A Survey](https://mlsanity.com/paper/2402.06330/) (2024)
- [Lifelong Learning on Evolving Graphs Under the Constraints of Imbalanced Classes and New Classes](https://mlsanity.com/paper/2112.10558/) (2021)
- [A Topology-aware Graph Coarsening Framework for Continual Graph Learning](https://mlsanity.com/paper/2401.03077/) (2024)
- [An Efficient Memory Module for Graph Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2411.06659/) (2024)
- [Robust Knowledge Adaptation for Dynamic Graph Neural Networks](https://mlsanity.com/paper/2207.10839/) (2022)
- [Universal Graph Continual Learning](https://mlsanity.com/paper/2308.13982/) (2023)
- [Continual Learning for Smart City: A Survey](https://mlsanity.com/paper/2404.00983/) (2024)
- [BeGin: Extensive Benchmark Scenarios and An Easy-to-use Framework for Graph Continual Learning](https://mlsanity.com/paper/2211.14568/) (2022)
- [On the Limitation and Experience Replay for GNNs in Continual Learning](https://mlsanity.com/paper/2302.03534/) (2023)
- [Learning Dynamic Graph Embeddings with Neural Controlled Differential Equations](https://mlsanity.com/paper/2302.11354/) (2023)
- [GRAFENNE: Learning on Graphs with Heterogeneous and Dynamic Feature Sets](https://mlsanity.com/paper/2306.03447/) (2023)
- [Dynamic Embedding Size Search with Minimum Regret for Streaming Recommender System](https://mlsanity.com/paper/2308.07760/) (2023)
- [Towards Continuous Reuse of Graph Models via Holistic Memory Diversification](https://mlsanity.com/paper/2406.07413/) (2024)

## Abstract (arXiv)

Graph neural networks (GNNs) have achieved strong performance in various applications. In the real world, network data is usually formed in a streaming fashion. The distributions of patterns that refer to neighborhood information of nodes may shift over time. The GNN model needs to learn the new patterns that cannot yet be captured. But learning incrementally leads to the catastrophic forgetting problem that historical knowledge is overwritten by newly learned knowledge. Therefore, it is important to train GNN model to learn new patterns and maintain existing patterns simultaneously, which few works focus on. In this paper, we propose a streaming GNN model based on continual learning so that the model is trained incrementally and up-to-date node representations can be obtained at each time step. Firstly, we design an approximation algorithm to detect new coming patterns efficiently based on information propagation. Secondly, we combine two perspectives of data replaying and model regularization for existing pattern consolidation. Specially, a hierarchy-importance sampling strategy for nodes is designed and a weighted regularization term for GNN parameters is derived, achieving greater stability and generalization of knowledge consolidation. Our model is evaluated on real and synthetic data sets and compared with multiple baselines. The results of node classification prove that our model can efficiently update model parameters and achieve comparable performance to model retraining. In addition, we also conduct a case study on the synthetic data, and carry out some specific analysis for each part of our model, illustrating its ability to learn new knowledge and maintain existing knowledge from different perspectives.