mlsanityData to 7 Oct 2026

ELDR: Expert-Locality-Aware Decode Routing for PD-Disaggregated MoE Serving

arXiv 2607.00466 ↗First posted 1 Jul 2026cs.DCcited by 2

On the map: Mixture-of-experts → Efficient MoE training and inference systems

also in 1 more
Abstract (arXiv)

In prefill-decode (PD) disaggregated LLM serving, each request is assigned to a decode worker after prefill. Existing decode routers balance only load; for mixture-of-experts (MoE) models this is incomplete: equally loaded workers can differ in latency, since each decode step loads the weights of every distinct expert its batch activates. We present ELDR, an expert-locality-aware decode router for PD-disaggregated MoE serving. From a request's prefill expert activations, ELDR builds an expert signature predicting the experts it will activate during generation. Offline, balanced K-means partitions signature space across decode workers; online, locality-band routing sends each request to the least-loaded worker among those best matching its signature. A signature cache, co-indexed with the KV cache at KV-block granularity, keeps signatures exact under prefix caching. Implemented in vLLM and evaluated on deployments of up to 40 GPUs, ELDR reduces median TPOT by 5.9-13.9% over the strongest of four load-balancing baselines across three MoE models and two workloads, with model outputs unchanged.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Efficient MoE training and inference systems, one of 12 directions in Mixture-of-experts. Builds on 25 of its 42 references that are in the map. First posted on arXiv 1 Jul 2026.

Builds on

25 of 42 references.

Closest ancestors · 6

Same direction, newest first

  1. 2026XShare: Collaborative in-Batch Expert Sharing for Faster MoE Inference6 Feb
  2. 2026Least-Loaded Expert Parallelism: Load Balancing An Imbalanced Mixture-of-Experts23 Jan
  3. 2025Efficient MoE Serving in the Memory-Bound Regime: Balance Activated Experts, Not Tokens10 Dec
  4. 2025Semantic Parallelism: Redefining Efficient MoE Inference via Model-Data Co-Scheduling6 Mar
  5. 2025MoETuner: Optimized Mixture of Expert Serving with Balanced Expert Placement and Token Routing10 Feb
  6. 2024Lynx: Enabling Efficient MoE Inference through Dynamic Batch-Aware Expert Selection13 Nov

Other references · 19

  1. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  3. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  4. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  5. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  6. 2021Program Synthesis with Large Language Modelscited by 3,509
  7. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,324
  8. 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,271
  9. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,107
  10. 2024OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problemscited by 1,099
  11. 2017Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problemscited by 642
  12. 2024WildChat: 1M ChatGPT Interaction Logs in the Wildcited by 598
7 more
  1. 2024DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Servingcited by 595
  2. 2024BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructionscited by 527
  3. 2022DS-1000: A Natural and Reliable Benchmark for Data Science Code Generationcited by 442
  4. 2024Mooncake: A KVCache-centric Disaggregated Architecture for LLM Servingcited by 193
  5. 2024Preble: Efficient Distributed Prompt Scheduling for LLM Servingcited by 68
  6. 2025Opportunistic Expert Activation: Batch-Aware Expert Routing for Faster Decode Without Retrainingcited by 6
  7. 2026Scaling Multi-Node Mixture-of-Experts Inference Using Expert Activation Patternscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Cacheable by Design? Training Mixture-of-Experts Routers for Locality Against the Edge Memory-Bandwidth Wall: A Pre-Registered Negative Result with a Systems Measurement Studycited by 1
  2. 2026SlimWise: Decoupling Expert Pruning Across Prefill and Decode for Efficient MoE Servingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026ExpertPlex: A High-Goodput Disaggregated Serving System for MoE LLMs with Adaptive Persistent Kernels
  2. 2026DECO: Sparse Mixture-of-Experts with Dense-Comparable Performance on End-Side Devices
  3. 2025MoE-Prism: Disentangling Monolithic Experts for Elastic MoE Services via Model-System Co-Designs
  4. 2026MoE Expert Execution in Disaggregated LLM Serving with a High-Bandwidth ReRAM Near-Memory Architecture
  5. 2026SpecPrefetch: Parameter-Efficient Expert Prefetching for Sparse MoE Foundation Models
  6. 2025ERMoE: Eigen-Reparameterized Mixture-of-Experts for Stable Routing and Interpretable Specialization
  7. 2026Efficient Mixture-of-Experts with Speculative Decoding via Expert Coactivation
  8. 2026AcceptMoE: Commitment-Weighted Self-Sizing Verifier Expert Sets for Efficient MoE Speculative Decoding