mlsanityData to 14 Oct 2026

FLEX-MoE: Federated Mixture-of-Experts with Load-balanced Expert Assignment for Edge Computing

arXiv 2512.23070 ↗First posted 28 Dec 2025cs.LGcited by 2

On the map: Mixture-of-experts → Federated mixture-of-experts

Abstract (arXiv)

Mixture-of-Experts (MoE) models enable scalable neural networks through conditional computation, offering enhanced effectiveness and efficiency for next-generation wireless communications. However, deploying MoE with federated learning (FL) over wireless and IoT edge networks faces two critical challenges: 1) resource-constrained clients cannot store large AI models with full expert sets, and 2) non-IID data distributions cause severe expert load imbalance that degrades model performance. To this end, we propose FLEX-MoE, a federated MoE framework that jointly optimizes expert assignment and load balancing under limited client capacity. Specifically, our approach introduces client-expert fitness scores that quantify expert suitability for local datasets through training feedback, and employs an optimization-based algorithm to maximize client-expert specialization while enforcing balanced expert utilization system-wide. Unlike greedy methods that focus solely on personalization while ignoring load imbalance, FLEX-MoE addresses expert utilization skew, which is particularly severe in heterogeneous edge FL. Our experimental results demonstrate superior accuracy and consistently balanced expert utilization across diverse resource-constrained scenarios for edge computing.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Federated mixture-of-experts, one of 12 directions in Mixture-of-experts. Builds on 17 of its 26 references that are in the map. First posted on arXiv 28 Dec 2025.

Builds on

17 of 26 references.

Closest ancestors · 7

Same direction, newest first

  1. 2025Efficient Training of Large-Scale AI Models Through Federated Mixture-of-Experts: A System-Level Approach8 Jul
  2. 2024FedMoE-DA: Federated Mixture of Experts via Domain Aware Fine-grained Aggregation4 Nov
  3. 2024FedMoE: Personalized Federated Learning via Heterogeneous Mixture of Experts21 Aug
  4. 2024pFedMoE: Data-Level Personalization with Mixture of Experts for Model-Heterogeneous Personalized Federated Learning2 Feb
  5. 2021Federated Mixture of Experts14 Jul
  6. 2020PFL-MoE: Personalized Federated Learning Based on Mixture of Experts31 Dec
  7. 2020Specialized federated learning using a mixture of experts5 Oct

Other references · 10

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  2. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,509
  3. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,073
  4. 2021Federated Learning on Non-IID Data: A Surveycited by 346
  5. 2024Auxiliary-Loss-Free Load Balancing Strategy for Mixture-of-Expertscited by 205
  6. 2023Sparse MoE as the New Dropout: Scaling Dense and Self-Slimmable Transformerscited by 59
  7. 2024WDMoE: Wireless Distributed Large Language Models with Mixture of Expertscited by 18
  8. 2024Mixture of Experts Made Personalized: Federated Prompt Learning for Vision-Language Modelscited by 17
  9. 2024Recurrent Early Exits for Federated Learning with Heterogeneous Clientscited by 12
  10. 2025Mixture-of-Experts for Distributed Edge Computing with Channel-Aware Gating Functioncited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026φ-Balancing for Mixture-of-Experts Trainingcited by 5
  2. 2026TAOT: Topology-Aware Optimal Transport for Dynamic Expert Replica Placement in MoE Trainingcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2025Towards Efficient Federated Learning of Networked Mixture-of-Experts for Mobile Edge Computing
  2. 2025CoMoE: Collaborative Optimization of Expert Aggregation and Offloading for MoE-based LLMs at Edge
  3. 2026HFedMoE: Resource-aware Heterogeneous Federated Learning with Mixture-of-Experts
  4. 2025Optimal Expert Selection for Distributed Mixture-of-Experts at the Wireless Edge
  5. 2025Stable-MoE: Lyapunov-based Token Routing for Distributed Mixture-of-Experts Training over Edge Networks
  6. 2025FLEx: Personalized Federated Learning for Mixture-of-Experts LLMs via Expert Grafting
  7. 2023Learning to Specialize: Joint Gating-Expert Training for Adaptive MoEs in Decentralized Settings
  8. 2026LAER-MoE: Load-Adaptive Expert Re-layout for Efficient Mixture-of-Experts Training