MoE-Prism: Disentangling Monolithic Experts for Elastic MoE Services via Model-System Co-Designs
On the map: Mixture-of-experts → Efficient MoE training and inference systems
Abstract (arXiv)
Mixture-of-Experts (MoE) scales model capacity through sparse activation, and is becoming an important architecture for large language models (LLMs). However, existing MoE serving systems typically execute all requests under a fixed routing configuration, limiting their ability to exploit heterogeneous computation requirements across requests. Routing top-k, which determines the number of routed experts activated per token, directly controls routed-expert computation and provides a natural mechanism for request-level compute elasticity. Realizing this capability, however, requires finer-grained routing units and efficient runtime execution for heterogeneous routing budgets. We present textscMoE-Prism, a model and system support framework for request-level compute elasticity in MoE serving. textscMoE-Prismdecomposes monolithic experts into fine-grained sub-experts to expose denser routing operating points and provides a k-aware serving runtime that effectively serves heterogeneous routing budgets under both throughput-oriented and latency-sensitive workloads. We implement textscMoE-Prism on top of vLLM and evaluate it on three representative MoE models. textscMoE-Prism expands the number of available routing operating points by 4×, improves offline inference throughput by up to 33.9%, and reduces online serving TTFT under heterogeneous workloads. These results demonstrate practical elastic MoE serving with request-level routing targets.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Efficient MoE training and inference systems, one of 12 directions in Mixture-of-experts. Builds on 34 of its 45 references that are in the map. First posted on arXiv 22 Oct 2025.
Builds on
34 of 45 references.
Closest ancestors · 2
Same direction, newest first
- 2025Elastic MoE: Unlocking the Inference-Time Scalability of Mixture-of-Experts26 Sep
- 2025D2MoE: Dual Routing and Dynamic Scheduling for Efficient On-Device MoE-based LLM Serving17 Apr
Other references · 32
- 2025Qwen3 Technical Reportcited by 8,195
- 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
- 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
- 2016Pointer Sentinel Mixture Modelscited by 2,992
- 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,140
- 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,838
- 2024Mixtral of Expertscited by 1,785
- 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,338
- 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,270
- 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,107
- 2023FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performancecited by 791
- 2017Crowdsourcing Multiple Choice Science Questionscited by 742
20 more
- 2024RouteLLM: Learning to Route LLMs with Preference Datacited by 633
- 2024QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMscited by 598
- 2024DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Servingcited by 595
- 2023Splitwise: Efficient generative LLM inference using phase splittingcited by 536
- 2025Kimi K2: Open Agentic Intelligencecited by 523
- 2024Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Servecited by 507
- 2023LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compressioncited by 443
- 2024LayerSkip: Enabling Early Exit Inference and Self-Speculative Decodingcited by 302
- 2022Confident Adaptive Language Modelingcited by 297
- 2024Llumnix: Dynamic Scheduling for Large Language Model Servingcited by 167
- 2024Not All Experts are Equal: Efficient Expert Pruning and Skipping for Mixture-of-Experts Large Language Modelscited by 140
- 2024LLaMA-MoE: Building Mixture-of-Experts from LLaMA with Continual Pre-trainingcited by 122
- 2025Mixture-of-Recursions: Learning Dynamic Recursive Depths for Adaptive Token-Level Computationcited by 120
- 2023Sparse MoE as the New Dropout: Scaling Dense and Self-Slimmable Transformerscited by 59
- 2024Dynamic Mixture of Experts: An Auto-Tuning Approach for Efficient Transformer Modelscited by 55
- 2024MuxServe: Flexible Spatial-Temporal Multiplexing for Multiple LLM Servingcited by 49
- 2024LLaMA-MoE v2: Exploring Sparsity of LLaMA from Perspective of Mixture-of-Experts with Post-Trainingcited by 25
- 2025Mixture of Neuron Expertscited by 7
- 2018Size-aware Sharding For Improving Tail Latencies in In-memory Key-value Storescited by 2
- 2025DualSparse-MoE: Coordinating Tensor/Neuron-Level Sparsity with Expert Partition and Reconstructioncited by 1
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026MoE Routing Testbed: Studying Expert Specialization and Routing Behavior at Small Scale
- 2025ElasticMoE: An Efficient Auto Scaling Method for Mixture-of-Experts Models
- 2026MoEless: Efficient MoE LLM Serving via Serverless Computing
- 2024Ada-K Routing: Boosting the Efficiency of MoE-based LLMs
- 2025DuoServe-MoE: Dual-Phase Expert Prefetch and Caching for LLM Inference QoS Assurance
- 2025Not All Models Suit Expert Offloading: On Local Routing Consistency of Mixture-of-Expert Models
- 2026ExpertPlex: A High-Goodput Disaggregated Serving System for MoE LLMs with Adaptive Persistent Kernels
- 2026CIPHER-MoE: Balancing Efficiency and Routing Fidelity in Trillion-Scale MoE Training