# The MoE-Empowered Edge LLMs Deployment: Architecture, Challenges, and Opportunities

arXiv: https://arxiv.org/abs/2502.08381 · HTML: https://mlsanity.com/paper/2502.08381/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Feb 2025.

## Built on by (most cited first, 5 of 5)

- [EC2MoE: Adaptive End-Cloud Pipeline Collaboration Enabling Scalable Mixture-of-Experts Inference](https://mlsanity.com/paper/2508.06024/) (2025)
- [SMoE: An Algorithm-System Co-Design for Pushing MoE to the Edge via Expert Substitution](https://mlsanity.com/paper/2508.18983/) (2025)
- [OrderMoE: An expert similarity driven distributed edge MoE inference](https://mlsanity.com/paper/2607.17154/) (2026)
- [OD-MoE: On-Demand Expert Loading for Cacheless Edge-Distributed MoE Inference](https://mlsanity.com/paper/2512.03927/) (2025)
- [Expert Streaming: Accelerating Low-Batch MoE Inference via Multi-chiplet Architecture and Dynamic Expert Trajectory Scheduling](https://mlsanity.com/paper/2603.27624/) (2026)

## Abstract (arXiv)

The powerfulness of LLMs indicates that deploying various LLMs with different scales and architectures on end, edge, and cloud to satisfy different requirements and adaptive heterogeneous hardware is the critical way to achieve ubiquitous intelligence for 6G. However, the massive parameter scale of LLMs poses significant challenges in deploying them on edge devices due to high computational and storage demands. Considering that the sparse activation in Mixture of Experts (MoE) is effective on scalable and dynamic allocation of computational and communications resources at the edge, this paper proposes a novel MoE-empowered collaborative deployment framework for edge LLMs, denoted as CoEL. This framework fully leverages the properties of MoE architecture and encompasses four key aspects: Perception, Deployment, Compression, and Updating. Edge servers broadcast their resource status and the specific resource requirements of LLMs to their neighbors. Then, utilizing this data, two sophisticated deployment strategies are proposed for satisfying varying model scales, ensuring that each model is deployed effectively. One for deploying LLMs on a single edge device through intra-device resource collaboration, and another for a distributed deployment across multiple edge devices via inter-device resource collaboration. Furthermore, both the models and the intermediate data are compressed for reducing memory footprint by quantization and reducing the volume of intermediate data by token fusion and pruning. Finally, given the dynamic of network topology, resource status, and user requirements, the deployment strategies are regularly updated to maintain its relevance and effectiveness. This paper also delineates the challenges and potential research directions for the deployment of edge LLMs.