mlsanityData to 14 Oct 2026

MAEM: Mitigating Recency Bias via a Replay-Free, Capacity-Controlled Dynamic Mixture of Experts for Class-Incremental Graph Learning

arXiv 2610.20238 ↗First posted 14 Oct 2026cs.LG stat.MLcited by 0

On the map: Mixture-of-experts → Mixture of LoRA experts also fits: Expert routing strategies

Abstract (arXiv)

Graph Neural Networks (GNNs) excel on graph-structured data but suffer catastrophic forgetting when classes arrive sequentially. In class-incremental graph learning, predictions are often dominated by recently learned classes, causing severe recency bias. Dynamic expansion can isolate knowledge across sessions, but many methods assume task-incremental settings and require task identities at test time to select branches or heads---information unavailable in class-incremental learning. Learnable routers offer an alternative, yet continual fine-tuning the router on new data can bias routing toward recent components. We propose MAEM, a replay-free, capacity-controlled dynamic mixture-of-experts framework for class-incremental graph learning. MAEM builds representations with Simplified Graph Convolution (SGC) and trains a dedicated expert for each new session, then freezes it to protect prior knowledge. For routing without task IDs or a shared trainable router, each expert uses a masked autoencoder (MAE); test nodes are assigned to the expert with the lowest reconstruction error, reducing routing-level recency bias. When the expert pool reaches a fixed capacity, MAEM merges the least-used experts via Git Re-Basin alignment and further distills them into a fused expert using topology-aware pseudo data synthesized from MAE distributions (DeepInversion-style), without storing or revisiting historical graph data. On four class-incremental graph benchmarks, MAEM mitigates recency bias, outperforms strong continual graph learning baselines, and keeps model capacity bounded under strict replay-free learning.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Mixture of LoRA experts, one of 12 directions in Mixture-of-experts. Builds on 30 of its 36 references that are in the map. First posted on arXiv 14 Oct 2026.

Builds on

30 of 36 references.

References · 30

  1. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
  2. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  3. 2017Graph Attention Networkscited by 8,923
  4. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  5. 2017Inductive Representation Learning on Large Graphscited by 6,320
  6. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  7. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,509
  8. 2016Learning without Forgettingcited by 2,496
  9. 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,333
  10. 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,163
  11. 2017Gradient Episodic Memory for Continual Learningcited by 1,898
  12. 2019Simplifying Graph Convolutional Networkscited by 1,477
18 more
  1. 2022Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference timecited by 1,396
  2. 2016Progressive Neural Networkscited by 1,361
  3. 2018Continual Lifelong Learning with Neural Networks: A Reviewcited by 1,330
  4. 2017Memory Aware Synapses: Learning what (not) to forgetcited by 1,105
  5. 2018Pitfalls of Graph Neural Network Evaluationcited by 1,007
  6. 2019Large Scale Incremental Learningcited by 732
  7. 2017Lifelong Learning with Dynamically Expandable Networkscited by 694
  8. 2022GraphMAE: Self-Supervised Masked Graph Autoencoderscited by 448
  9. 2019Dreaming to Distill: Data-free Knowledge Transfer via DeepInversioncited by 423
  10. 2016Expert Gate: Lifelong Learning with a Network of Expertscited by 372
  11. 2017Deep Gaussian Embedding of Graphs: Unsupervised Inductive Learning via Rankingcited by 335
  12. 2020Wiki-CS: A Wikipedia-Based Benchmark for Graph Neural Networkscited by 260
  13. 2019Data-Free Learning of Student Networkscited by 204
  14. 2019Zero-Shot Knowledge Distillation in Deep Networkscited by 121
  15. 2023Graph Mixture of Experts: Learning on Large-Scale Graphs with Explicit Diversity Modelingcited by 64
  16. 2024Graph Continual Learning with Debiased Lossless Memory Replaycited by 9
  17. 2025Dynamic Mixture-of-Experts for Incremental Graph Learningcited by 1
  18. 2026TAAM:Inductive Graph-Class Incremental Learning with Task-Aware Adaptive Modulationcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Task-Aware Adaptive Modulation: A Replay-Free and Resource-Efficient Approach For Continual Graph Learning
  2. 2026Stable Routing for Mixture-of-Experts in Class-Incremental Learning
  3. 2025GMoPE:A Prompt-Expert Mixture Framework for Graph Foundation Models
  4. 2025Adaptive Graph Mixture of Residual Experts: Unsupervised Learning on Diverse Graphs with Heterogeneous Specialization
  5. 2024An Efficient Memory Module for Graph Few-Shot Class-Incremental Learning
  6. 2026Generalizing GNNs with Tokenized Mixture of Experts
  7. 2025HERO: Preserving Structure and Semantics in Heterogeneous Continual Graph Learning
  8. 2024Graph Unlearning with Efficient Partial Retraining