# Overcoming Catastrophic Forgetting in Graph Neural Networks

arXiv: https://arxiv.org/abs/2012.06002 · HTML: https://mlsanity.com/paper/2012.06002/

## Key facts

- Cited by 95 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Dec 2020.

## Built on by (most cited first, 30 of 95)

- [Scaling & Shifting Your Features: A New Baseline for Efficient Model Tuning](https://mlsanity.com/paper/2210.08823/) (2022)
- [Dataset Distillation via Factorization](https://mlsanity.com/paper/2210.16774/) (2022)
- [Universal Prompt Tuning for Graph Neural Networks](https://mlsanity.com/paper/2209.15240/) (2022)
- [Deep Model Reassembly](https://mlsanity.com/paper/2210.17409/) (2022)
- [Structure-free Graph Condensation: From Large-scale Graphs to Condensed Graph-free Data](https://mlsanity.com/paper/2306.02664/) (2023)
- [Contrastive Model Inversion for Data-Free Knowledge Distillation](https://mlsanity.com/paper/2105.08584/) (2021)
- [A Survey on Graph Representation Learning Methods](https://mlsanity.com/paper/2204.01855/) (2022)
- [CaT: Balanced Continual Graph Learning with Graph Condensation](https://mlsanity.com/paper/2309.09455/) (2023)
- [Text-space Graph Foundation Models: Comprehensive Benchmarks and New Insights](https://mlsanity.com/paper/2406.10727/) (2024)
- [Continual Learning on Dynamic Graphs via Parameter Isolation](https://mlsanity.com/paper/2305.13825/) (2023)
- [GraphControl: Adding Conditional Control to Universal Graph Pre-trained Models for Graph Domain Transfer Learning](https://mlsanity.com/paper/2310.07365/) (2023)
- [Self-Supervised Continual Graph Learning in Adaptive Riemannian Spaces](https://mlsanity.com/paper/2211.17068/) (2022)
- [Graph Lifelong Learning: A Survey](https://mlsanity.com/paper/2202.10688/) (2022)
- [Learning with Recoverable Forgetting](https://mlsanity.com/paper/2207.08224/) (2022)
- [Learning Graph Neural Networks for Image Style Transfer](https://mlsanity.com/paper/2207.11681/) (2022)
- [Attention Diversification for Domain Generalization](https://mlsanity.com/paper/2210.04206/) (2022)
- [Hierarchical Prototype Networks for Continual Graph Representation Learning](https://mlsanity.com/paper/2111.15422/) (2021)
- [Meta-Aggregator: Learning to Aggregate for 1-bit Graph Neural Networks](https://mlsanity.com/paper/2109.12872/) (2021)
- [Reinforced Continual Learning for Graphs](https://mlsanity.com/paper/2209.01556/) (2022)
- [Continual Learning on Graphs: Challenges, Solutions, and Opportunities](https://mlsanity.com/paper/2402.11565/) (2024)
- [Machine Unlearning: A Survey](https://mlsanity.com/paper/2306.03558/) (2023)
- [Continual Graph Learning: A Survey](https://mlsanity.com/paper/2301.12230/) (2023)
- [Towards Robust Graph Incremental Learning on Evolving Graphs](https://mlsanity.com/paper/2402.12987/) (2024)
- [Graph Learning under Distribution Shifts: A Comprehensive Survey on Domain Adaptation, Out-of-distribution, and Continual Learning](https://mlsanity.com/paper/2402.16374/) (2024)
- [SHARE: An SLM-based Hierarchical Action CorREction Assistant for Text-to-SQL](https://mlsanity.com/paper/2506.00391/) (2025)
- [Point2Seq: Detecting 3D Objects as Sequences](https://mlsanity.com/paper/2203.13394/) (2022)
- [Memorization in deep learning: A survey](https://mlsanity.com/paper/2406.03880/) (2024)
- [Topology-aware Embedding Memory for Continual Learning on Expanding Networks](https://mlsanity.com/paper/2401.13200/) (2024)
- [DSLR: Diversity Enhancement and Structure Learning for Rehearsal-based Graph Continual Learning](https://mlsanity.com/paper/2402.13711/) (2024)
- [Towards Open Temporal Graph Neural Networks](https://mlsanity.com/paper/2303.15015/) (2023)

## Abstract (arXiv)

Catastrophic forgetting refers to the tendency that a neural network "forgets" the previous learned knowledge upon learning new tasks. Prior methods have been focused on overcoming this problem on convolutional neural networks (CNNs), where the input samples like images lie in a grid domain, but have largely overlooked graph neural networks (GNNs) that handle non-grid data. In this paper, we propose a novel scheme dedicated to overcoming catastrophic forgetting problem and hence strengthen continual learning in GNNs. At the heart of our approach is a generic module, termed as topology-aware weight preserving~(TWP), applicable to arbitrary form of GNNs in a plug-and-play fashion. Unlike the main stream of CNN-based continual learning methods that rely on solely slowing down the updates of parameters important to the downstream task, TWP explicitly explores the local structures of the input graph, and attempts to stabilize the parameters playing pivotal roles in the topological aggregation. We evaluate TWP on different GNN backbones over several datasets, and demonstrate that it yields performances superior to the state of the art. Code is publicly available at urlhttps://github.com/hhliu79/TWP.