mlsanityData to 14 Oct 2026

FedCoE: Bridging Generalization and Personalization via Federated Coordinated Dual-level MoEs

arXiv 2605.21264 ↗First posted 20 May 2026cs.LGcited by 0

On the map: Mixture-of-experts → Federated mixture-of-experts

Abstract (arXiv)

Federated Learning (FL) has emerged as a promising paradigm for privacy-preserving distributed learning. However, existing FL methods face a fundamental challenge. Traditional averaging-based approaches suffer from parameter divergence under non-IID conditions, while personalized FL methods overfit to local data and fail to generalize to new clients (cold-start problem). Mixture-of-Experts naturally addresses this by routing heterogeneous data to specialized experts rather than forcing uniform aggregation. In this paper, we propose FedCoE, a Federated Coordinated dual-level mixture-of-Experts framework that effectively balances global generalization with local personalization. FedCoE maintains multiple independent global expert models on the server and employs a shared gating network to dynamically model client-expert correlations during aggregation, effectively mitigating expert drift and gating inconsistency. To address the cold-start challenge, we introduce an adaptive mechanism that enables new clients to immediately leverage the global expert pool without extensive local training. Extensive experiments demonstrate that FedCoE achieves 78.00% global accuracy and 89.32% personalized accuracy on average, outperforming the baseline by 8.82% and 29.19%, respectively. In cold-start scenarios, FedCoE delivers 77.27% accuracy without any local fine-tuning, outperforming baselines by over 12.54%.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Federated mixture-of-experts, one of 12 directions in Mixture-of-experts. Builds on 24 of its 35 references that are in the map. First posted on arXiv 20 May 2026.

Builds on

24 of 35 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025PM-MOE: Mixture of Experts on Private Model Parameters for Personalized Federated Learning1 Feb
  2. 2024FedMoE-DA: Federated Mixture of Experts via Domain Aware Fine-grained Aggregation4 Nov
  3. 2024FedMoE: Personalized Federated Learning via Heterogeneous Mixture of Experts21 Aug
  4. 2024pFedMoE: Data-Level Personalization with Mixture of Experts for Model-Heterogeneous Personalized Federated Learning2 Feb
  5. 2023Learning to Specialize: Joint Gating-Expert Training for Adaptive MoEs in Decentralized Settings14 Jun

Other references · 19

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  3. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,509
  4. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,073
  5. 2019Advances and Open Problems in Federated Learningcited by 3,046
  6. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  7. 2020GShard: Scaling Giant Models with Conditional Computation and Automatic Shardingcited by 1,631
  8. 2021Scaling Vision with Sparse Mixture of Expertscited by 747
  9. 2021Federated Learning on Non-IID Data Silos: An Experimental Studycited by 568
  10. 2019Federated Learning with Personalization Layerscited by 541
  11. 2022DeepSpeed-MoE: Advancing Mixture-of-Experts Inference and Training to Power Next-Generation AI Scalecited by 404
  12. 2020Think Locally, Act Globally: Federated Learning with Local and Global Representationscited by 325
7 more
  1. 2020Personalized Federated Learning: A Meta-Learning Approachcited by 312
  2. 2020Personalized Federated Learning with First Order Model Optimizationcited by 201
  3. 2021FedKD: Communication Efficient Federated Learning via Knowledge Distillationcited by 187
  4. 2023Towards Personalized Federated Learning via Heterogeneous Model Reassemblycited by 31
  5. 2023Robust Mixture-of-Expert Training for Convolutional Neural Networkscited by 30
  6. 2023PeFLL: Personalized Federated Learning by Learning to Learncited by 21
  7. 2024Personalized Federated Learning with Mixture of Models for Adaptive Prediction and Model Fine-Tuningcited by 8

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2020Specialized federated learning using a mixture of experts
  2. 2022Adaptive Expert Models for Personalization in Federated Learning
  3. 2020PFL-MoE: Personalized Federated Learning Based on Mixture of Experts
  4. 2025FedMerge: Federated Personalization via Model Merging
  5. 2025FLEx: Personalized Federated Learning for Mixture-of-Experts LLMs via Expert Grafting
  6. 2026Aggregation Alignment for Federated Learning with Mixture-of-Experts under Data Heterogeneity
  7. 2026Global Federated Learning Strategies for Building Efficient Personalized Models
  8. 2021Personalised Federated Learning: A Combinational Approach