# Hierarchical Prototype Networks for Continual Graph Representation Learning

arXiv: https://arxiv.org/abs/2111.15422 · HTML: https://mlsanity.com/paper/2111.15422/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 40 of its 70 references that are in the map.
- First posted on arXiv 30 Nov 2021.

## Builds on (most cited first, 30 of 40)

- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [How Powerful are Graph Neural Networks?](https://mlsanity.com/paper/1810.00826/) (2018)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Learning without Forgetting](https://mlsanity.com/paper/1606.09282/) (2016)
- [Gradient Episodic Memory for Continual Learning](https://mlsanity.com/paper/1706.08840/) (2017)
- [DeepWalk: Online Learning of Social Representations](https://mlsanity.com/paper/1403.6652/) (2014)
- [Progressive Neural Networks](https://mlsanity.com/paper/1606.04671/) (2016)
- [Continual Learning with Deep Generative Replay](https://mlsanity.com/paper/1705.08690/) (2017)
- [Memory Aware Synapses: Learning what (not) to forget](https://mlsanity.com/paper/1711.09601/) (2017)
- [Simple and Deep Graph Convolutional Networks](https://mlsanity.com/paper/2007.02133/) (2020)
- [Geom-GCN: Geometric Graph Convolutional Networks](https://mlsanity.com/paper/2002.05287/) (2020)
- [Large Scale Incremental Learning](https://mlsanity.com/paper/1905.13260/) (2019)
- [Lifelong Learning with Dynamically Expandable Networks](https://mlsanity.com/paper/1708.01547/) (2017)
- [DropEdge: Towards Deep Graph Convolutional Networks on Node Classification](https://mlsanity.com/paper/1907.10903/) (2019)
- [Three scenarios for continual learning](https://mlsanity.com/paper/1904.07734/) (2019)
- [FastGCN: Fast Learning with Graph Convolutional Networks via Importance Sampling](https://mlsanity.com/paper/1801.10247/) (2018)
- [Gradient based sample selection for online continual learning](https://mlsanity.com/paper/1903.08671/) (2019)
- [DeepGCNs: Can GCNs Go as Deep as CNNs?](https://mlsanity.com/paper/1904.03751/) (2019)
- [Orthogonal Gradient Descent for Continual Learning](https://mlsanity.com/paper/1910.07104/) (2019)
- [Gradient Projection Memory for Continual Learning](https://mlsanity.com/paper/2103.09762/) (2021)
- [Supermasks in Superposition](https://mlsanity.com/paper/2006.14769/) (2020)
- [ViTAE: Vision Transformer Advanced by Exploring Intrinsic Inductive Bias](https://mlsanity.com/paper/2106.03348/) (2021)
- [Layer-Dependent Importance Sampling for Training Deep and Large Graph Convolutional Networks](https://mlsanity.com/paper/1911.07323/) (2019)
- [Scalable and Order-robust Continual Learning with Additive Parameter Decomposition](https://mlsanity.com/paper/1902.09432/) (2019)
- [Meta-GNN: On Few-shot Node Classification in Graph Meta-learning](https://mlsanity.com/paper/1905.09718/) (2019)
- [Less-forgetting Learning in Deep Neural Networks](https://mlsanity.com/paper/1607.00122/) (2016)
- [Overcoming Catastrophic Forgetting in Graph Neural Networks](https://mlsanity.com/paper/2012.06002/) (2020)

## Built on by (most cited first, 28 of 28)

- [CaT: Balanced Continual Graph Learning with Graph Condensation](https://mlsanity.com/paper/2309.09455/) (2023)
- [Self-Supervised Continual Graph Learning in Adaptive Riemannian Spaces](https://mlsanity.com/paper/2211.17068/) (2022)
- [Continual Learning on Graphs: Challenges, Solutions, and Opportunities](https://mlsanity.com/paper/2402.11565/) (2024)
- [Continual Graph Learning: A Survey](https://mlsanity.com/paper/2301.12230/) (2023)
- [Towards Robust Graph Incremental Learning on Evolving Graphs](https://mlsanity.com/paper/2402.12987/) (2024)
- [Graph Learning under Distribution Shifts: A Comprehensive Survey on Domain Adaptation, Out-of-distribution, and Continual Learning](https://mlsanity.com/paper/2402.16374/) (2024)
- [Replay-and-Forget-Free Graph Class-Incremental Learning: A Task Profiling and Prompting Approach](https://mlsanity.com/paper/2410.10341/) (2024)
- [An Efficient Memory Module for Graph Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2411.06659/) (2024)
- [PUMA: Efficient Continual Graph Learning for Node Classification with Graph Condensation](https://mlsanity.com/paper/2312.14439/) (2023)
- [Continual Learning for Smart City: A Survey](https://mlsanity.com/paper/2404.00983/) (2024)
- [GCAL: Adapting Graph Models to Evolving Domain Shifts](https://mlsanity.com/paper/2505.16860/) (2025)
- [On the Limitation and Experience Replay for GNNs in Continual Learning](https://mlsanity.com/paper/2302.03534/) (2023)
- [Towards Continuous Reuse of Graph Models via Holistic Memory Diversification](https://mlsanity.com/paper/2406.07413/) (2024)
- [Graph Memory Learning: Imitating Lifelong Remembering and Forgetting of Brain Networks](https://mlsanity.com/paper/2407.19183/) (2024)
- [Efficient and Robust Continual Graph Learning for Graph Classification in Biology](https://mlsanity.com/paper/2411.11668/) (2024)
- [HPCR: Holistic Proxy-based Contrastive Replay for Online Continual Learning](https://mlsanity.com/paper/2309.15038/) (2023)
- [Fairness in Augmented Graph Learning: A Survey](https://mlsanity.com/paper/2504.21296/) (2025)
- [Learning System Dynamics without Forgetting](https://mlsanity.com/paper/2407.00717/) (2024)
- [Prompt-Driven Continual Graph Learning](https://mlsanity.com/paper/2502.06327/) (2025)
- [When LLMs meet open-world graph learning: a new perspective for unlabeled data uncertainty](https://mlsanity.com/paper/2505.13989/) (2025)
- [HERO: Preserving Structure and Semantics in Heterogeneous Continual Graph Learning](https://mlsanity.com/paper/2505.17458/) (2025)
- [G2LoRA: Gradient Orthogonal Low-Rank Adaptation Framework for Graph Continual Learning on Text-Attributed Graphs](https://mlsanity.com/paper/2606.01873/) (2026)
- [Learning Continually on a Sequence of Graphs -- The Dynamical System Way](https://mlsanity.com/paper/2305.12030/) (2023)
- [Adapt before Continual Learning](https://mlsanity.com/paper/2506.03956/) (2025)
- [Continual Learning for fMRI-Based Brain Disorder Diagnosis via Functional Connectivity Matrices Generative Replay](https://mlsanity.com/paper/2604.14259/) (2026)
- [UFO: A Unified Flow-Oriented Framework for Robust Continual Graph Learning](https://mlsanity.com/paper/2605.09862/) (2026)
- [DRIFT: A Benchmark for Task-Free Continual Graph Learning with Continuous Distribution Shifts](https://mlsanity.com/paper/2605.12998/) (2026)
- [SAOT: Self-Supervised Continual Graph Learning with Structure-Aware Optimal Transport](https://mlsanity.com/paper/2607.00377/) (2026)

## Abstract (arXiv)

Despite significant advances in graph representation learning, little attention has been paid to the more practical continual learning scenario in which new categories of nodes (e.g., new research areas in citation networks, or new types of products in co-purchasing networks) and their associated edges are continuously emerging, causing catastrophic forgetting on previous categories. Existing methods either ignore the rich topological information or sacrifice plasticity for stability. To this end, we present Hierarchical Prototype Networks (HPNs) which extract different levels of abstract knowledge in the form of prototypes to represent the continuously expanded graphs. Specifically, we first leverage a set of Atomic Feature Extractors (AFEs) to encode both the elemental attribute information and the topological structure of the target node. Next, we develop HPNs to adaptively select relevant AFEs and represent each node with three levels of prototypes. In this way, whenever a new category of nodes is given, only the relevant AFEs and prototypes at each level will be activated and refined, while others remain uninterrupted to maintain the performance over existing nodes. Theoretically, we first demonstrate that the memory consumption of HPNs is bounded regardless of how many tasks are encountered. Then, we prove that under mild constraints, learning new tasks will not alter the prototypes matched to previous data, thereby eliminating the forgetting problem. The theoretical results are supported by experiments on five datasets, showing that HPNs not only outperform state-of-the-art baseline techniques but also consume relatively less memory.