mlsanityData to 7 Oct 2026

Federated mixture-of-experts

37 papers0 this weekHigh confidence in this grouping: the papers form one clear topicHigh confidence in this grouping: the papers form one clear topic · About

Keywords: federated, federated learning, clients, client, local

Key facts. Federated mixture-of-experts: 37 papers on arXiv since 2020 (data through 7 Oct 2026). 12 of its 37 mapped papers (32%) cite “PFL-MoE: Personalized Federated Learning Based on Mixture of Experts” (2020, arXiv 2012.15589). 12 of its 37 mapped papers (32%) cite “FedMoE: Personalized Federated Learning via Heterogeneous Mixture of Experts” (2024, arXiv 2408.11304).

What’s new

Nothing new since 17 Sep. Latest: 11 Sep 2026.

Foundations

Most cited by papers here

  1. PFL-MoE: Personalized Federated Learning Based on Mixture of Experts2020cited by 12 here
  2. FedMoE: Personalized Federated Learning via Heterogeneous Mixture of Experts2024cited by 12 here
  3. FedMoE-DA: Federated Mixture of Experts via Domain Aware Fine-grained Aggregation2024cited by 9 here
  4. Specialized federated learning using a mixture of experts2020cited by 7 here
  5. Federated Mixture of Experts2021cited by 7 here

Lineage

The papers this direction builds on, oldest first.

Each paper on the line cites the one before.

  1. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 10 here
  2. 2020GShard: Scaling Giant Models with Conditional Computation and Automatic Shardingcited by 2 here
  3. 2022Mixture-of-Experts with Expert Choice Routingcited by 3 here
  4. 2024DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Modelscited by 4 here
  5. 2024DeepSeek-V3 Technical Reportcited by 4 here
19 more it builds on, not in this chain
  1. 2020Specialized federated learning using a mixture of expertscited by 7 here
  2. 2020PFL-MoE: Personalized Federated Learning Based on Mixture of Expertscited by 12 here
  3. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 12 here
  4. 2021Federated Mixture of Expertscited by 7 here
  5. 2022DeepSpeed-MoE: Advancing Mixture-of-Experts Inference and Training to Power Next-Generation AI Scalecited by 2 here
  6. 2023Learning to Specialize: Joint Gating-Expert Training for Adaptive MoEs in Decentralized Settingscited by 3 here
  7. 2023SiDA-MoE: Sparsity-Inspired Data-Aware Serving for Efficient and Scalable Large Mixture-of-Experts Modelscited by 3 here
  8. 2024Mixtral of Expertscited by 3 here
  9. 2024pFedMoE: Data-Level Personalization with Mixture of Experts for Model-Heterogeneous Personalized Federated Learningcited by 7 here
  10. 2024Not All Experts are Equal: Efficient Expert Pruning and Skipping for Mixture-of-Experts Large Language Modelscited by 4 here
  11. 2024FedMoE: Personalized Federated Learning via Heterogeneous Mixture of Expertscited by 12 here
  12. 2024OLMoE: Open Mixture-of-Experts Language Modelscited by 3 here
  13. 2024Mixture of Experts Made Personalized: Federated Prompt Learning for Vision-Language Modelscited by 3 here
  14. 2024FedMoE-DA: Federated Mixture of Experts via Domain Aware Fine-grained Aggregationcited by 9 here
  15. 2025PM-MOE: Mixture of Experts on Private Model Parameters for Personalized Federated Learningcited by 6 here
  16. 2025dFLMoE: Decentralized Federated Learning via Mixture of Experts for Medical Data Analysiscited by 3 here
  17. 2025Mixture-of-Experts for Distributed Edge Computing with Channel-Aware Gating Functioncited by 3 here
  18. 2025Efficient Training of Large-Scale AI Models Through Federated Mixture-of-Experts: A System-Level Approachcited by 3 here
  19. 2025FFT-MoE: Efficient Federated Fine-Tuning for Foundation Models via Large-scale Sparse MoE under Heterogeneous Edgecited by 3 here

The 24 most cited of 82.

Browse

30 of 37, ranked 6–35

  1. 6pFedMoE: Data-Level Personalization with Mixture of Experts for Model-Heterogeneous Personalized Federated Learning20247
  2. 7PM-MOE: Mixture of Experts on Private Model Parameters for Personalized Federated Learning20256
  3. 8dFLMoE: Decentralized Federated Learning via Mixture of Experts for Medical Data Analysis20253
  4. 9Efficient Training of Large-Scale AI Models Through Federated Mixture-of-Experts: A System-Level Approach20253
  5. 10FFT-MoE: Efficient Federated Fine-Tuning for Foundation Models via Large-scale Sparse MoE under Heterogeneous Edge20253
  6. 11Learning to Specialize: Joint Gating-Expert Training for Adaptive MoEs in Decentralized Settings20233
  7. 12HFedMoE: Resource-aware Heterogeneous Federated Learning with Mixture-of-Experts20262
  8. 13FLEx: Personalized Federated Learning for Mixture-of-Experts LLMs via Expert Grafting20252
  9. 14Adaptive Expert Models for Personalization in Federated Learning20222
  10. 15Wireless Federated Multi-Task LLM Fine-Tuning via Sparse-and-Orthogonal LoRA20261
All 30
  1. 16Aggregation Alignment for Federated Learning with Mixture-of-Experts under Data Heterogeneity20261
  2. 17FedALT: Federated Fine-Tuning through Adaptive Local Training with Rest-of-World LoRA20251
  3. 18Robust Federated Learning by Mixture of Experts20211
  4. 19Mixture of Predefined Experts: Maximizing Data Usage on Vertical Federated Learning20260
  5. 20DeepFusion: Accelerating MoE Training via Federated Knowledge Distillation from Heterogeneous Edge Devices20260
  6. 21DriftGuard: Mitigating Asynchronous Data Drift in Federated Learning20260
  7. 22UB-SMoE: Universally Balanced Sparse Mixture-of-Experts for Resource-adaptive Federated Fine-tuning of Foundation Models20260
  8. 23FedCoE: Bridging Generalization and Personalization via Federated Coordinated Dual-level MoEs20260
  9. 24Conflict-Aware Federated Fine-Tuning of Large Language Models with Mixture-of-Experts20260
  10. 25Fisher-Routed Mixture of Experts for Federated Class-Incremental Learning20260
  11. 26FedGuide: Diffusion Prior Alignment and Value Baseline Guidance for Heterogeneous Federated Reinforcement Learning20260
  12. 27FedWeave: Rethinking the Unit of Specialization in Heterogeneous Federated MoE-LoRA20260
  13. 28MoE Router-Guided Clustering for Heterogeneous Federated Instruction Tuning20260
  14. 29Task-Aware Federated Fine-Tuning for MoE-based Large Language Models20260
  15. 30FedMerge: Federated Personalization via Model Merging20250
  16. 31Mosaic: Data-Free Knowledge Distillation via Mixture-of-Experts for Heterogeneous Distributed Environments20250
  17. 32FLAME: Towards Federated Fine-Tuning Large Language Models Through Adaptive SMoE20250
  18. 33Shift Happens: Mixture of Experts based Continual Adaptation in Federated Learning20250
  19. 34Closer to Reality: Practical Semi-Supervised Federated Learning for Foundation Model Adaptation20250
  20. 35Federated Fine-Tuning of Sparsely-Activated Large Language Models on Resource-Constrained Devices20250