mlsanityData to 7 Oct 2026

Fisher-Routed Mixture of Experts for Federated Class-Incremental Learning

arXiv 2606.28835 ↗First posted 27 Jun 2026cs.LG cs.AIcited by 0

On the map: Mixture-of-experts → Federated mixture-of-experts

Abstract (arXiv)

Federated Learning (FL) emerged as a promising distributed machine learning paradigm. However, extending FL to the class incremental learning scenarios introduces unique challenges: 1) Capacity conflict and catastrophic forgetting from the shared model overloading, 2) Heterogeneity from Non-Independent and Identically Distributed (Non-IID) data, and 3) Synchronized class misalignment. In this paper, we propose Fisher-Routed MiXture of Experts for Federated Class-Incremental Learning (textscFedFMX), a novel framework to address these challenges via adaptive expert specialization across clients. The crucial insight is to route each sample to an expert subset that jointly optimizes knowledge acquisition and retention. Specifically, we introduce a Fisher-Routed Expert Scoring (FRES) module to estimate expert importance via Fisher-based stability cost and gradient-based plasticity gain. Then, we design an Adaptive Expert Selection (AES) module by quantifying marginal contributions for adaptive expert subset determination. Finally, by the routing-aware regularization (RAR), we achieve load balance and efficient FL training. We theoretically prove the O(T-1) convergence rate. Extensive experiments on multiple benchmarks compared with state-of-the-art methods demonstrate the superiority of textscFedFMX.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Federated mixture-of-experts, one of 12 directions in Mixture-of-experts. Builds on 46 of its 71 references that are in the map. First posted on arXiv 27 Jun 2026.

Builds on

46 of 71 references.

Closest ancestors · 7

Same direction, newest first

  1. 2025Efficient Training of Large-Scale AI Models Through Federated Mixture-of-Experts: A System-Level Approach8 Jul
  2. 2025dFLMoE: Decentralized Federated Learning via Mixture of Experts for Medical Data Analysis13 Mar
  3. 2025PM-MOE: Mixture of Experts on Private Model Parameters for Personalized Federated Learning1 Feb
  4. 2024FedMoE-DA: Federated Mixture of Experts via Domain Aware Fine-grained Aggregation4 Nov
  5. 2024FedMoE: Personalized Federated Learning via Heterogeneous Mixture of Experts21 Aug
  6. 2024pFedMoE: Data-Level Personalization with Mixture of Experts for Model-Heterogeneous Personalized Federated Learning2 Feb
  7. 2023Learning to Specialize: Joint Gating-Expert Training for Adaptive MoEs in Decentralized Settings14 Jun

Other references · 39

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  4. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  5. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  6. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
  7. 2019Advances and Open Problems in Federated Learningcited by 3,021
  8. 2016Learning without Forgettingcited by 2,465
  9. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,531
  10. 2018Moment Matching for Multi-Source Domain Adaptationcited by 1,327
  11. 2022Mixture-of-Experts with Expert Choice Routingcited by 611
  12. 2021Federated Learning on Non-IID Data Silos: An Experimental Studycited by 566
27 more
  1. 2018Sparsified SGD with Memorycited by 373
  2. 2018Recent Advances in Open Set Recognition: A Surveycited by 309
  3. 2019Error Feedback Fixes SignSGD and other Gradient Compression Schemescited by 306
  4. 2022On the Representation Collapse of Sparse Mixture of Expertscited by 156
  5. 2022Federated Class-Incremental Learningcited by 126
  6. 2025A Comprehensive Survey of Mixture-of-Experts: Algorithms, Theory, and Applicationscited by 91
  7. 2023TARGET: Federated Class-Continual Learning via Exemplar-Free Distillationcited by 64
  8. 2023No One Left Behind: Real-World Federated Class-Incremental Learningcited by 53
  9. 2023Federated Continual Learning via Knowledge Fusion: A Surveycited by 53
  10. 2025Demons in the Detail: On Implementing Load Balancing Loss for Training Specialized Mixture-of-Expert Modelscited by 53
  11. 2023A Data-Free Approach to Mitigate Catastrophic Forgetting in Federated Class Incremental Learning for Vision Taskscited by 41
  12. 2024Towards Efficient Replay in Federated Incremental Learningcited by 33
  13. 2025CL-LoRA: Continual Low-Rank Adaptation for Rehearsal-Free Class-Incremental Learningcited by 26
  14. 2022FedCross: Towards Accurate Federated Learning via Multi-Model Cross-Aggregationcited by 21
  15. 2025Ten Challenging Problems in Federated Foundation Modelscited by 20
  16. 2024PILoRA: Prototype Guided Incremental LoRA for Federated Class-Incremental Learningcited by 19
  17. 2025FedVLA: Federated Vision-Language-Action Learning with Dual Gating Mixture-of-Experts for Robotic Manipulationcited by 18
  18. 2024Text-Enhanced Data-free Approach for Federated Class-Incremental Learningcited by 16
  19. 2024Rehearsal-Free Continual Federated Learning with Synergistic Synaptic Intelligencecited by 15
  20. 2024FedProK: Trustworthy Federated Class-Incremental Learning via Prototypical Feature Knowledge Transfercited by 9
  21. 2025Federated Class-Incremental Learning: A Hybrid Approach Using Latent Exemplars and Data-Free Techniques to Address Local and Global Forgettingcited by 9
  22. 2025Class-wise Balancing Data Replay for Federated Class-Incremental Learningcited by 9
  23. 2025SacFL: Self-Adaptive Federated Continual Learning for Resource-Constrained End Devicescited by 8
  24. 2025Resource-Constrained Federated Continual Learning: What Does Matter?cited by 5
  25. 2024Rehearsal-free Federated Domain-incremental Learningcited by 4
  26. 2025Autoencoder-Based Hybrid Replay for Class-Incremental Learningcited by 3
  27. 2026FGGM: Fisher-Guided Gradient Masking for Continual Learningcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Stable Routing for Mixture-of-Experts in Class-Incremental Learning
  2. 2026PRISM: Exposing and Resolving Spurious Isolation in Federated Multimodal Continual Learning
  3. 2026UB-SMoE: Universally Balanced Sparse Mixture-of-Experts for Resource-adaptive Federated Fine-tuning of Foundation Models
  4. 2021Federated Mixture of Experts
  5. 2025FLAME: Towards Federated Fine-Tuning Large Language Models Through Adaptive SMoE
  6. 2026HFedMoE: Resource-aware Heterogeneous Federated Learning with Mixture-of-Experts
  7. 2024FedDr+: Stabilizing Dot-regression with Global Feature Distillation for Federated Learning
  8. 2022Adaptive Expert Models for Personalization in Federated Learning