# Stable-MoE: Lyapunov-based Token Routing for Distributed Mixture-of-Experts Training over Edge Networks

arXiv: https://arxiv.org/abs/2512.06784 · HTML: https://mlsanity.com/paper/2512.06784/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Expert routing strategies, one of 12 directions in Mixture-of-experts.
- Builds on 13 of its 25 references that are in the map.
- First posted on arXiv 7 Dec 2025.

## Placed in

- Mixture-of-experts / [Expert routing strategies](https://mlsanity.com/moe/expert-routing-strategies/)

## Builds on (most cited first, 13 of 13)

- [Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer](https://mlsanity.com/paper/1701.06538/) (2017)
- [Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity](https://mlsanity.com/paper/2101.03961/) (2021)
- [GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding](https://mlsanity.com/paper/2006.16668/) (2020)
- [GLaM: Efficient Scaling of Language Models with Mixture-of-Experts](https://mlsanity.com/paper/2112.06905/) (2021)
- [Mixture-of-Experts with Expert Choice Routing](https://mlsanity.com/paper/2202.09368/) (2022)
- [BASE Layers: Simplifying Training of Large, Sparse Models](https://mlsanity.com/paper/2103.16716/) (2021)
- [WDMoE: Wireless Distributed Mixture of Experts for Large Language Models](https://mlsanity.com/paper/2411.06681/) (2024)
- [Communication-Efficient Distributed On-Device LLM Inference Over Wireless Networks](https://mlsanity.com/paper/2503.14882/) (2025)
- [Mixture-of-Experts for Distributed Edge Computing with Channel-Aware Gating Function](https://mlsanity.com/paper/2504.00819/) (2025)
- [The Security and Privacy of Mobile Edge Computing: An Artificial Intelligence Perspective](https://mlsanity.com/paper/2401.01589/) (2024)
- [SMILE: Scaling Mixture-of-Experts with Efficient Bi-level Routing](https://mlsanity.com/paper/2212.05191/) (2022)
- [When MoE Meets Blockchain: A Trustworthy Distributed Framework of Large Models](https://mlsanity.com/paper/2509.12141/) (2025)
- [Federated Attention: A Distributed Paradigm for Collaborative LLM Inference over Edge Networks](https://mlsanity.com/paper/2511.02647/) (2025)

## Built on by (most cited first, 1 of 1)

- [CR^2: Cost-Aware Risk-Controlled Routing for Wireless Device-Edge LLM Inference](https://mlsanity.com/paper/2605.12001/) (2026)

## Abstract (arXiv)

The sparse activation mechanism of mixture of experts (MoE) model empowers edge intelligence with enhanced training efficiency and reduced computational resource consumption. However, traditional token routing in distributed MoE training faces significant challenges in resource-constrained edge networks characterized by heterogeneous computing capabilities and stochastic token arrivals, which inevitably suffer from workload backlog, resource inefficiency, and performance degradation. To address this issue, we propose a novel Lyapunov-based token routing framework for distributed MoE training over resource-heterogeneous edge networks, termed Stable-MoE. Specifically, we formulate a stochastic optimization problem to maximize both system throughput and gating consistency via optimizing the token routing strategy and computational resource allocation, while ensuring long-term stability of both token and energy queues at the edge devices. Using the Lyapunov optimization, we transform the intractable long-term optimization problem into tractable per-slot subproblems by enabling online decision-making of token routing and computation frequency utilization without the knowledge of future system states. Experimental results on the SVHN and CIFAR-100 datasets demonstrate that Stable-MoE outperforms the baselines with at least 40% and 5% gains in system throughput and test accuracy, respectively.