# CaT: Balanced Continual Graph Learning with Graph Condensation

arXiv: https://arxiv.org/abs/2309.09455 · HTML: https://mlsanity.com/paper/2309.09455/

## Key facts

- Cited by 52 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 42 references that are in the map.
- First posted on arXiv 18 Sep 2023.

## Builds on (most cited first, 30 of 33)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [How Powerful are Graph Neural Networks?](https://mlsanity.com/paper/1810.00826/) (2018)
- [Learning without Forgetting](https://mlsanity.com/paper/1606.09282/) (2016)
- [Open Graph Benchmark: Datasets for Machine Learning on Graphs](https://mlsanity.com/paper/2005.00687/) (2020)
- [Gradient Episodic Memory for Continual Learning](https://mlsanity.com/paper/1706.08840/) (2017)
- [Simplifying Graph Convolutional Networks](https://mlsanity.com/paper/1902.07153/) (2019)
- [Memory Aware Synapses: Learning what (not) to forget](https://mlsanity.com/paper/1711.09601/) (2017)
- [A Comprehensive Survey of Continual Learning: Theory, Method and Application](https://mlsanity.com/paper/2302.00487/) (2023)
- [Gated Graph Sequence Neural Networks](https://mlsanity.com/paper/1511.05493/) (2015)
- [Dataset Condensation with Gradient Matching](https://mlsanity.com/paper/2006.05929/) (2020)
- [Dataset Condensation with Distribution Matching](https://mlsanity.com/paper/2110.04181/) (2021)
- [Dataset Condensation with Differentiable Siamese Augmentation](https://mlsanity.com/paper/2102.08259/) (2021)
- [Dataset Distillation](https://mlsanity.com/paper/1811.10959/) (2018)
- [Class-Incremental Learning: A Survey](https://mlsanity.com/paper/2302.03648/) (2023)
- [Graph Condensation for Graph Neural Networks](https://mlsanity.com/paper/2110.07580/) (2021)
- [Generalizing Dataset Distillation via Deep Generative Prior](https://mlsanity.com/paper/2305.01649/) (2023)
- [Condensing Graphs via One-Step Gradient Matching](https://mlsanity.com/paper/2206.07746/) (2022)
- [Overcoming Catastrophic Forgetting in Graph Neural Networks](https://mlsanity.com/paper/2012.06002/) (2020)
- [Graph Condensation via Receptive Field Distribution Matching](https://mlsanity.com/paper/2206.13697/) (2022)
- [GraphSAIL: Graph Structure Aware Incremental Learning for Recommender Systems](https://mlsanity.com/paper/2008.13517/) (2020)
- [Distilled Replay: Overcoming Forgetting through Synthetic Samples](https://mlsanity.com/paper/2103.15851/) (2021)
- [Continual Learning of Knowledge Graph Embeddings](https://mlsanity.com/paper/2101.05850/) (2021)
- [Sample Condensation in Online Continual Learning](https://mlsanity.com/paper/2206.11849/) (2022)
- [Graph Condensation for Inductive Node Representation Learning](https://mlsanity.com/paper/2307.15967/) (2023)
- [Hierarchical Prototype Networks for Continual Graph Representation Learning](https://mlsanity.com/paper/2111.15422/) (2021)
- [Reducing catastrophic forgetting with learning on synthetic data](https://mlsanity.com/paper/2004.14046/) (2020)

## Built on by (most cited first, 30 of 52)

- [A Comprehensive Survey on Graph Reduction: Sparsification, Coarsening, and Condensation](https://mlsanity.com/paper/2402.03358/) (2024)
- [Navigating Complexity: Toward Lossless Graph Condensation via Expanding Window Matching](https://mlsanity.com/paper/2402.05011/) (2024)
- [Graph Condensation: A Survey](https://mlsanity.com/paper/2401.11720/) (2024)
- [Rethinking and Accelerating Graph Condensation: A Training-Free Approach with Class Partition](https://mlsanity.com/paper/2405.13707/) (2024)
- [Continual Learning on Graphs: Challenges, Solutions, and Opportunities](https://mlsanity.com/paper/2402.11565/) (2024)
- [Topology-aware Embedding Memory for Continual Learning on Expanding Networks](https://mlsanity.com/paper/2401.13200/) (2024)
- [Replay-and-Forget-Free Graph Class-Incremental Learning: A Task Profiling and Prompting Approach](https://mlsanity.com/paper/2410.10341/) (2024)
- [An Efficient Memory Module for Graph Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2411.06659/) (2024)
- [PUMA: Efficient Continual Graph Learning for Node Classification with Graph Condensation](https://mlsanity.com/paper/2312.14439/) (2023)
- [A Survey on Graph Condensation](https://mlsanity.com/paper/2402.02000/) (2024)
- [Graph Continual Learning with Debiased Lossless Memory Replay](https://mlsanity.com/paper/2404.10984/) (2024)
- [Graph Condensation for Open-World Graph Learning](https://mlsanity.com/paper/2405.17003/) (2024)
- [Two Trades is not Baffled: Condensing Graph via Crafting Rational Gradient Matching](https://mlsanity.com/paper/2402.04924/) (2024)
- [RobGC: Towards Robust Graph Condensation](https://mlsanity.com/paper/2406.13200/) (2024)
- [GCAL: Adapting Graph Models to Evolving Domain Shifts](https://mlsanity.com/paper/2505.16860/) (2025)
- [Improve Cross-Architecture Generalization on Dataset Distillation](https://mlsanity.com/paper/2402.13007/) (2024)
- [Towards Continuous Reuse of Graph Models via Holistic Memory Diversification](https://mlsanity.com/paper/2406.07413/) (2024)
- [GC4NC: A Benchmark Framework for Graph Condensation on Node Classification with New Insights](https://mlsanity.com/paper/2406.16715/) (2024)
- [Recent Advances in Malware Detection: Graph Learning and Explainability](https://mlsanity.com/paper/2502.10556/) (2025)
- [CondTSF: One-line Plugin of Dataset Condensation for Time Series Forecasting](https://mlsanity.com/paper/2406.02131/) (2024)
- [Dataset Condensation with Latent Quantile Matching](https://mlsanity.com/paper/2406.09860/) (2024)
- [Contrastive Graph Condensation: Advancing Data Versatility through Self-Supervised Learning](https://mlsanity.com/paper/2411.17063/) (2024)
- [Scalable Graph Condensation with Evolving Capabilities](https://mlsanity.com/paper/2502.17614/) (2025)
- [GCondenser: Benchmarking Graph Condensation](https://mlsanity.com/paper/2405.14246/) (2024)
- [Backdoor Graph Condensation](https://mlsanity.com/paper/2407.11025/) (2024)
- [Federated Continual Graph Learning](https://mlsanity.com/paper/2411.18919/) (2024)
- [Simple yet Effective Graph Distillation via Clustering](https://mlsanity.com/paper/2505.20807/) (2025)
- [Inductive Graph Few-shot Class Incremental Learning](https://mlsanity.com/paper/2411.06634/) (2024)
- [Efficient and Robust Continual Graph Learning for Graph Classification in Biology](https://mlsanity.com/paper/2411.11668/) (2024)
- [Online Continual Graph Learning](https://mlsanity.com/paper/2508.03283/) (2025)

## Abstract (arXiv)

Continual graph learning (CGL) is purposed to continuously update a graph model with graph data being fed in a streaming manner. Since the model easily forgets previously learned knowledge when training with new-coming data, the catastrophic forgetting problem has been the major focus in CGL. Recent replay-based methods intend to solve this problem by updating the model using both (1) the entire new-coming data and (2) a sampling-based memory bank that stores replayed graphs to approximate the distribution of historical data. After updating the model, a new replayed graph sampled from the incoming graph will be added to the existing memory bank. Despite these methods are intuitive and effective for the CGL, two issues are identified in this paper. Firstly, most sampling-based methods struggle to fully capture the historical distribution when the storage budget is tight. Secondly, a significant data imbalance exists in terms of the scales of the complex new-coming graph data and the lightweight memory bank, resulting in unbalanced training. To solve these issues, a Condense and Train (CaT) framework is proposed in this paper. Prior to each model update, the new-coming graph is condensed to a small yet informative synthesised replayed graph, which is then stored in a Condensed Graph Memory with historical replay graphs. In the continual learning phase, a Training in Memory scheme is used to update the model directly with the Condensed Graph Memory rather than the whole new-coming graph, which alleviates the data imbalance problem. Extensive experiments conducted on four benchmark datasets successfully demonstrate superior performances of the proposed CaT framework in terms of effectiveness and efficiency. The code has been released on https://github.com/superallen13/CaT-CGL.