# FLEX-MoE: Federated Mixture-of-Experts with Load-balanced Expert Assignment for Edge Computing

arXiv: https://arxiv.org/abs/2512.23070 · HTML: https://mlsanity.com/paper/2512.23070/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Federated mixture-of-experts, one of 12 directions in Mixture-of-experts.
- Builds on 17 of its 26 references that are in the map.
- First posted on arXiv 28 Dec 2025.

## Placed in

- Mixture-of-experts / [Federated mixture-of-experts](https://mlsanity.com/moe/federated-mixture-of-experts/)

## Builds on (most cited first, 17 of 17)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer](https://mlsanity.com/paper/1701.06538/) (2017)
- [Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity](https://mlsanity.com/paper/2101.03961/) (2021)
- [Federated Learning on Non-IID Data: A Survey](https://mlsanity.com/paper/2106.06843/) (2021)
- [Auxiliary-Loss-Free Load Balancing Strategy for Mixture-of-Experts](https://mlsanity.com/paper/2408.15664/) (2024)
- [Sparse MoE as the New Dropout: Scaling Dense and Self-Slimmable Transformers](https://mlsanity.com/paper/2303.01610/) (2023)
- [Federated Mixture of Experts](https://mlsanity.com/paper/2107.06724/) (2021)
- [FedMoE: Personalized Federated Learning via Heterogeneous Mixture of Experts](https://mlsanity.com/paper/2408.11304/) (2024)
- [pFedMoE: Data-Level Personalization with Mixture of Experts for Model-Heterogeneous Personalized Federated Learning](https://mlsanity.com/paper/2402.01350/) (2024)
- [Specialized federated learning using a mixture of experts](https://mlsanity.com/paper/2010.02056/) (2020)
- [PFL-MoE: Personalized Federated Learning Based on Mixture of Experts](https://mlsanity.com/paper/2012.15589/) (2020)
- [WDMoE: Wireless Distributed Large Language Models with Mixture of Experts](https://mlsanity.com/paper/2405.03131/) (2024)
- [Mixture of Experts Made Personalized: Federated Prompt Learning for Vision-Language Models](https://mlsanity.com/paper/2410.10114/) (2024)
- [Recurrent Early Exits for Federated Learning with Heterogeneous Clients](https://mlsanity.com/paper/2405.14791/) (2024)
- [FedMoE-DA: Federated Mixture of Experts via Domain Aware Fine-grained Aggregation](https://mlsanity.com/paper/2411.02115/) (2024)
- [Mixture-of-Experts for Distributed Edge Computing with Channel-Aware Gating Function](https://mlsanity.com/paper/2504.00819/) (2025)
- [Efficient Training of Large-Scale AI Models Through Federated Mixture-of-Experts: A System-Level Approach](https://mlsanity.com/paper/2507.05685/) (2025)

## Built on by (most cited first, 2 of 2)

- [φ-Balancing for Mixture-of-Experts Training](https://mlsanity.com/paper/2605.15403/) (2026)
- [TAOT: Topology-Aware Optimal Transport for Dynamic Expert Replica Placement in MoE Training](https://mlsanity.com/paper/2608.03676/) (2026)

## Abstract (arXiv)

Mixture-of-Experts (MoE) models enable scalable neural networks through conditional computation, offering enhanced effectiveness and efficiency for next-generation wireless communications. However, deploying MoE with federated learning (FL) over wireless and IoT edge networks faces two critical challenges: 1) resource-constrained clients cannot store large AI models with full expert sets, and 2) non-IID data distributions cause severe expert load imbalance that degrades model performance. To this end, we propose FLEX-MoE, a federated MoE framework that jointly optimizes expert assignment and load balancing under limited client capacity. Specifically, our approach introduces client-expert fitness scores that quantify expert suitability for local datasets through training feedback, and employs an optimization-based algorithm to maximize client-expert specialization while enforcing balanced expert utilization system-wide. Unlike greedy methods that focus solely on personalization while ignoring load imbalance, FLEX-MoE addresses expert utilization skew, which is particularly severe in heterogeneous edge FL. Our experimental results demonstrate superior accuracy and consistently balanced expert utilization across diverse resource-constrained scenarios for edge computing.