mlsanityData to 14 Oct 2026

PM-MOE: Mixture of Experts on Private Model Parameters for Personalized Federated Learning

arXiv 2502.00354 ↗First posted 1 Feb 2025cs.LG cs.AI cs.CRcited by 9

On the map: Mixture-of-experts → Federated mixture-of-experts

Abstract (arXiv)

Federated learning (FL) has gained widespread attention for its privacy-preserving and collaborative learning capabilities. Due to significant statistical heterogeneity, traditional FL struggles to generalize a shared model across diverse data domains. Personalized federated learning addresses this issue by dividing the model into a globally shared part and a locally private part, with the local model correcting representation biases introduced by the global model. Nevertheless, locally converged parameters more accurately capture domain-specific knowledge, and current methods overlook the potential benefits of these parameters. To address these limitations, we propose PM-MoE architecture. This architecture integrates a mixture of personalized modules and an energy-based personalized modules denoising, enabling each client to select beneficial personalized parameters from other clients. We applied the PM-MoE architecture to nine recent model-split-based personalized federated learning algorithms, achieving performance improvements with minimal additional training. Extensive experiments on six widely adopted datasets and two heterogeneity settings validate the effectiveness of our approach. The source code is available at urlhttps://github.com/dannis97500/PM-MOE.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Placed in Federated mixture-of-experts, one of 12 directions in Mixture-of-experts. Builds on 29 of its 63 references that are in the map. First posted on arXiv 1 Feb 2025.

Builds on

29 of 63 references.

Closest ancestors · 1

Same direction, newest first

  1. 2020PFL-MoE: Personalized Federated Learning Based on Mixture of Experts31 Dec

Other references · 28

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2023Segment Anythingcited by 8,234
  3. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  4. 2019Advances and Open Problems in Federated Learningcited by 3,046
  5. 2015Character-level Convolutional Networks for Text Classificationcited by 2,330
  6. 2020Energy-based Out-of-distribution Detectioncited by 1,299
  7. 2016Bag of Tricks for Efficient Text Classificationcited by 876
  8. 2021Model-Contrastive Federated Learningcited by 766
  9. 2020Ditto: Fair and Robust Federated Learning Through Personalizationcited by 672
  10. 2020Personalized Federated Learning with Moreau Envelopescited by 618
  11. 2021FedBN: Federated Learning on Non-IID Features via Local Batch Normalizationcited by 571
  12. 2019Federated Learning with Personalization Layerscited by 541
16 more
  1. 2021Exploiting Shared Representations for Personalized Federated Learningcited by 541
  2. 2021Data-Free Knowledge Distillation for Heterogeneous Federated Learningcited by 383
  3. 2021FedProto: Federated Prototype Learning across Heterogeneous Clientscited by 352
  4. 2020Think Locally, Act Globally: Federated Learning with Local and Global Representationscited by 325
  5. 2022Why Can GPT Learn In-Context? Language Models Implicitly Perform Gradient Descent as Meta-Optimizerscited by 204
  6. 2021FedBABU: Towards Enhanced Representation for Federated Image Classificationcited by 182
  7. 2020FedVision: An Online Visual Object Detection Platform Powered by Federated Learningcited by 124
  8. 2022FedALA: Adaptive Local Aggregation for Personalized Federated Learningcited by 122
  9. 2023FedGH: Heterogeneous Federated Learning with Generalized Global Headercited by 56
  10. 2021Adapt to Adaptation: Learning Personalization for Cross-Silo Federated Learningcited by 53
  11. 2022Isolation and Impartial Aggregation: A Paradigm of Incremental Learning without Interferencecited by 44
  12. 2023Eliminating Domain Bias for Federated Learning in Representation Spacecited by 38
  13. 2023FedCP: Separating Feature Information for Personalized Federated Learning via Conditional Policycited by 36
  14. 2020Federated Learning and Differential Privacy: Software tools analysis, the Sherpa.ai FL framework and methodological guidelines for preserving data privacycited by 22
  15. 2022FedREP: Towards Horizontal Federated Load Forecasting for Retail Energy Providerscited by 14
  16. 2022FedGen: Generalizable Federated Learning for Sequential Datacited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 7.

9 papers.

  1. 2026HFedMoE: Resource-aware Heterogeneous Federated Learning with Mixture-of-Expertscited by 11
  2. 2026Aggregation Alignment for Federated Learning with Mixture-of-Experts under Data Heterogeneitycited by 4
  3. 2025FLAME: Towards Federated Fine-Tuning Large Language Models Through Adaptive SMoEcited by 0
  4. 2025Aligning by Misaligning: Boundary-aware Curriculum Learning for Multimodal Alignmentcited by 0
  5. 2026DeepFusion: Accelerating MoE Training via Federated Knowledge Distillation from Heterogeneous Edge Devicescited by 0
  6. 2026FedCoE: Bridging Generalization and Personalization via Federated Coordinated Dual-level MoEscited by 0
  7. 2026Conflict-Aware Federated Fine-Tuning of Large Language Models with Mixture-of-Expertscited by 0
  8. 2026Fisher-Routed Mixture of Experts for Federated Class-Incremental Learningcited by 0
  9. 2026Does Every User Need a Private LoRA? Decoupling Personalization from Per-User Adaptationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025A Novel Algorithm for Personalized Federated Learning: Knowledge Distillation with Weighted Combination Loss
  2. 2020Specialized federated learning using a mixture of experts
  3. 2022Adaptive Expert Models for Personalization in Federated Learning
  4. 2024FedMoE-DA: Federated Mixture of Experts via Domain Aware Fine-grained Aggregation
  5. 2024Towards Personalized Federated Learning via Comprehensive Knowledge Distillation
  6. 2022A Generative Framework for Personalized Learning and Estimation: Theory, Algorithms, and Privacy
  7. 2025FLEx: Personalized Federated Learning for Mixture-of-Experts LLMs via Expert Grafting
  8. 2021Personalised Federated Learning: A Combinational Approach