mlsanityData to 7 Oct 2026

Efficient MoE Inference with Fine-Grained Scheduling of Disaggregated Expert Parallelism

arXiv 2512.21487 ↗First posted 25 Dec 2025cs.DC cs.AIcited by 1

On the map: Mixture-of-experts → Efficient MoE training and inference systems

also in 1 more
Abstract (arXiv)

The mixture-of-experts (MoE) architecture scales model size with sublinear computational increase but suffers from memory-intensive inference due to KV caches and sparse expert activation. Recent disaggregated expert parallelism (DEP) distributes attention and experts to dedicated GPU groups but lacks support for shared experts and efficient task scheduling, limiting performance. We propose FinDEP, a fine-grained task scheduling algorithm for DEP that maximizes task overlap to improve MoE inference throughput. FinDEP introduces three innovations: 1) partitioning computation/communication into smaller tasks for fine-grained pipelining, 2) formulating a scheduling optimization supporting variable granularity and ordering, and 3) developing an efficient solver for this large search space. Experiments on four GPU systems with DeepSeek-V2 and Qwen3-MoE show FinDEP improves throughput by up to 1.61x over prior methods, achieving up to 1.24x speedup on a 32-GPU system.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Efficient MoE training and inference systems, one of 12 directions in Mixture-of-experts. Builds on 19 of its 33 references that are in the map. First posted on arXiv 25 Dec 2025.

Builds on

19 of 33 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025Step-3 is Large yet Affordable: Model-system Co-design for Cost-effective Decoding25 Jul
  2. 2025FSMoE: A Flexible and Scalable Training System for Sparse Mixture-of-Experts Models18 Jan
  3. 2024MoE-Lightning: High-Throughput MoE Inference on Memory-constrained GPUs18 Nov
  4. 2024Fiddler: CPU-GPU Orchestration for Fast Inference of Mixture-of-Experts Models10 Feb
  5. 2023Fast Inference of Mixture-of-Experts Language Models with Offloading28 Dec

Other references · 14

  1. 2017Attention Is All You Needcited by 59,932
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  4. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
  5. 2020GShard: Scaling Giant Models with Conditional Computation and Automatic Shardingcited by 1,601
  6. 2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Modelcited by 1,093
  7. 2024DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Modelscited by 834
  8. 2024DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Servingcited by 595
  9. 2025FlashInfer: Efficient and Customizable Attention Engine for LLM Inference Servingcited by 248
  10. 2023PowerInfer: Fast Large Language Model Serving with a Consumer-grade GPUcited by 210
  11. 2023FlexMoE: Scaling Large-scale Sparse Pre-trained Model Training via Dynamic Device Placementcited by 52
  12. 2020Automatic Horizontal Fusion for GPU Kernelscited by 17
2 more
  1. 2025Injecting Adrenaline into LLM Serving: Boosting Resource Utilization and Throughput via Attention Disaggregationcited by 14
  2. 2024Task Scheduling for Efficient Inference of Large Language Models on Single Moderate GPU Systemscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026AFD-Ledger: Deployment Provisioning for Attention--FFN Disaggregationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026A Scheduling Framework for Efficient MoE Inference on Edge GPU-NDP Systems
  2. 2024DAOP: Data-Aware Offloading and Predictive Pre-Calculation for Efficient MoE Inference
  3. 2026TriMoE: Augmenting GPU with AMX-Enabled CPU and DIMM-NDP for High-Throughput MoE Inference via Offloading
  4. 2025ExpertFlow: Adaptive Expert Scheduling and Memory Coordination for Efficient MoE Inference
  5. 2025Accelerating MoE Model Inference with Expert Sharding
  6. 2025HybriMoE: Hybrid CPU-GPU Scheduling and Cache Management for Efficient MoE Inference
  7. 2025HarMoEny: Efficient Multi-GPU Inference of MoE Models
  8. 2025OD-MoE: On-Demand Expert Loading for Cacheless Edge-Distributed MoE Inference