MAEM: Mitigating Recency Bias via a Replay-Free, Capacity-Controlled Dynamic Mixture of Experts for Class-Incremental Graph Learning
On the map: Mixture-of-experts → Mixture of LoRA experts also fits: Expert routing strategies
Abstract (arXiv)
Graph Neural Networks (GNNs) excel on graph-structured data but suffer catastrophic forgetting when classes arrive sequentially. In class-incremental graph learning, predictions are often dominated by recently learned classes, causing severe recency bias. Dynamic expansion can isolate knowledge across sessions, but many methods assume task-incremental settings and require task identities at test time to select branches or heads---information unavailable in class-incremental learning. Learnable routers offer an alternative, yet continual fine-tuning the router on new data can bias routing toward recent components. We propose MAEM, a replay-free, capacity-controlled dynamic mixture-of-experts framework for class-incremental graph learning. MAEM builds representations with Simplified Graph Convolution (SGC) and trains a dedicated expert for each new session, then freezes it to protect prior knowledge. For routing without task IDs or a shared trainable router, each expert uses a masked autoencoder (MAE); test nodes are assigned to the expert with the lowest reconstruction error, reducing routing-level recency bias. When the expert pool reaches a fixed capacity, MAEM merges the least-used experts via Git Re-Basin alignment and further distills them into a fused expert using topology-aware pseudo data synthesized from MAE distributions (DeepInversion-style), without storing or revisiting historical graph data. On four class-incremental graph benchmarks, MAEM mitigates recency bias, outperforms strong continual graph learning baselines, and keeps model capacity bounded under strict replay-free learning.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Mixture of LoRA experts, one of 12 directions in Mixture-of-experts. Builds on 30 of its 36 references that are in the map. First posted on arXiv 14 Oct 2026.
Builds on
30 of 36 references.
References · 30
- 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
- 2015Distilling the Knowledge in a Neural Networkcited by 10,324
- 2017Graph Attention Networkscited by 8,923
- 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
- 2017Inductive Representation Learning on Large Graphscited by 6,320
- 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
- 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,509
- 2016Learning without Forgettingcited by 2,496
- 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,333
- 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,163
- 2017Gradient Episodic Memory for Continual Learningcited by 1,898
- 2019Simplifying Graph Convolutional Networkscited by 1,477
18 more
- 2022Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference timecited by 1,396
- 2016Progressive Neural Networkscited by 1,361
- 2018Continual Lifelong Learning with Neural Networks: A Reviewcited by 1,330
- 2017Memory Aware Synapses: Learning what (not) to forgetcited by 1,105
- 2018Pitfalls of Graph Neural Network Evaluationcited by 1,007
- 2019Large Scale Incremental Learningcited by 732
- 2017Lifelong Learning with Dynamically Expandable Networkscited by 694
- 2022GraphMAE: Self-Supervised Masked Graph Autoencoderscited by 448
- 2019Dreaming to Distill: Data-free Knowledge Transfer via DeepInversioncited by 423
- 2016Expert Gate: Lifelong Learning with a Network of Expertscited by 372
- 2017Deep Gaussian Embedding of Graphs: Unsupervised Inductive Learning via Rankingcited by 335
- 2020Wiki-CS: A Wikipedia-Based Benchmark for Graph Neural Networkscited by 260
- 2019Data-Free Learning of Student Networkscited by 204
- 2019Zero-Shot Knowledge Distillation in Deep Networkscited by 121
- 2023Graph Mixture of Experts: Learning on Large-Scale Graphs with Explicit Diversity Modelingcited by 64
- 2024Graph Continual Learning with Debiased Lossless Memory Replaycited by 9
- 2025Dynamic Mixture-of-Experts for Incremental Graph Learningcited by 1
- 2026TAAM:Inductive Graph-Class Incremental Learning with Task-Aware Adaptive Modulationcited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Task-Aware Adaptive Modulation: A Replay-Free and Resource-Efficient Approach For Continual Graph Learning
- 2026Stable Routing for Mixture-of-Experts in Class-Incremental Learning
- 2025GMoPE:A Prompt-Expert Mixture Framework for Graph Foundation Models
- 2025Adaptive Graph Mixture of Residual Experts: Unsupervised Learning on Diverse Graphs with Heterogeneous Specialization
- 2024An Efficient Memory Module for Graph Few-Shot Class-Incremental Learning
- 2026Generalizing GNNs with Tokenized Mixture of Experts
- 2025HERO: Preserving Structure and Semantics in Heterogeneous Continual Graph Learning
- 2024Graph Unlearning with Efficient Partial Retraining