mlsanityData to 14 Oct 2026

DuoRate: Two-Rate Per-Expert Precision Allocation for Mixture-of-Experts Serving

arXiv 2610.13328 ↗First posted 9 Oct 2026cs.LGcited by 0

On the map: Mixture-of-experts → Expert routing strategies also fits: Efficient MoE training and inference systems

Abstract (arXiv)

Serving a Mixture-of-Experts (MoE) model pays for expert precision twice: the resident footprint pays for every expert the router can reach, while each batch-one decode step pays only for the experts it selects. These are two different rate functionals over the same per-expert widths, yet published per-expert MoE allocators constrain only the first. We call the difference the store-read gap; in the released BitsMoE checkpoint, 2.54 bits per weight (bpw) of storage become 7.50 bpw of decode traffic. We propose DuoRate, which stores once and reads per workload. Offline, it stores one uniform tier of a prefix-readable weight stream and bakes a distortion tensor. Online, it profiles a workload's own prefill without labels and solves a Lagrangian rate-distortion allocation under the decode functional; routing frequency provably cancels from every per-expert decision, and the solve costs one dot product per expert and a bisection. A deployment therefore keeps a single artifact that prefill reads at the stored rate and decode reads at the workload's own rate. On Qwen3-30B-A3B, DuoRate reading 2.3 bpw exceeds every 2-bit-class baseline by 6.3 to 47.0 points of average accuracy, and at 2.9 bpw it matches 4-bit formats that read 1.4-1.6 bpw more. The advantage is largest under long decoding: on GSM8K, DuoRate at 2.9 bpw matches or exceeds every 4-bit format and trails BF16 by only 0.25 points, and at 2.6 bpw it still matches or exceeds the 4.5-bit NF4 and FP4. Against controls in its own format, the allocation alone is worth 0.07-0.11 bpw of read rate. Code will be released upon publication.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Expert routing strategies, one of 12 directions in Mixture-of-experts. Builds on 38 of its 43 references that are in the map. First posted on arXiv 9 Oct 2026.

Builds on

38 of 43 references.

References · 38

  1. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  2. 2025Qwen3 Technical Reportcited by 8,820
  3. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  4. 2016Pointer Sentinel Mixture Modelscited by 3,096
  5. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
  6. 2023SqueezeLLM: Dense-and-Sparse Quantizationcited by 294
  7. 2019HAWQ-V2: Hessian Aware trace-Weighted Quantization of Neural Networkscited by 261
  8. 2023EdgeMoE: Empowering Sparse Large Language Models on Mobile Devicescited by 69
  9. 2024HOBBIT: A Mixed Precision Expert Offloading System for Fast MoE Inferencecited by 55
  10. 2024Any-Precision LLM: Low-Cost Deployment of Multiple, Different-Sized LLMscited by 54
  11. 2024Mixture Compressor for Mixture-of-Experts LLMs Gains Morecited by 41
  12. 2025MoEQuant: Enhancing Quantization for Mixture-of-Experts Large Language Models via Expert-Balanced Sampling and Affinity Guidancecited by 34
26 more
  1. 2024Accuracy is Not All You Needcited by 31
  2. 2025MxMoE: Mixed-precision Quantization for MoE with Accuracy and Performance Co-Designcited by 30
  3. 2025D2MoE: Dual Routing and Dynamic Scheduling for Efficient On-Device MoE-based LLM Servingcited by 20
  4. 2024Progressive Mixed-Precision Decoding for Efficient LLM Inferencecited by 18
  5. 2025Matryoshka Quantizationcited by 14
  6. 2026HiFloat4 Format for Language Model Inferencecited by 11
  7. 2025AnyBCQ: Hardware Efficient Flexible Binary-Coded Quantization for Multi-Precision LLMscited by 9
  8. 2025MiLo: Efficient Quantized MoE Inference with Mixture of Low-Rank Compensatorscited by 8
  9. 2025Dynamic Expert Quantization for Scalable Mixture-of-Experts Inferencecited by 7
  10. 2026GEMQ: Global Expert-Level Mixed-Precision Quantization for MoE LLMscited by 7
  11. 2026AlphaQ: Calibration-Free Bit Allocation for Mixture-of-Experts Quantizationcited by 7
  12. 2025Radio: Rate-Distortion Optimization for Large Language Model Compressioncited by 5
  13. 2025MoPEQ: Mixture of Mixed Precision Quantized Expertscited by 5
  14. 2025DuoServe-MoE: Dual-Phase Expert Prefetch and Caching for LLM Inference QoS Assurancecited by 5
  15. 2025From Tokens to Layers: Redefining Stall-Free Scheduling for MoE Serving with Layered Prefillcited by 4
  16. 2026Mix-Quant: Quantized Prefilling, Precise Decoding for Agentic LLMscited by 4
  17. 2025SliceMoE: Bit-Sliced Expert Caching under Miss-Rate Constraints for Efficient MoE Inferencecited by 3
  18. 2026RateQuant: Optimal Mixed-Precision KV Cache Quantization via Rate-Distortion Theorycited by 3
  19. 2026ELDR: Expert-Locality-Aware Decode Routing for PD-Disaggregated MoE Servingcited by 3
  20. 2026Recurrent Residual Quantization: A Progressive Multi-Precision Representation for LLMscited by 3
  21. 2025MC#: Mixture Compressor for Mixture-of-Experts Large Modelscited by 2
  22. 2026MoE-Prefill: Zero Redundancy Overheads in MoE Prefill Servingcited by 2
  23. 2026PagedWeight: Efficient MoE LLM Serving with Dynamic Quality-Aware Weight Quantizationcited by 2
  24. 2026PrismQuant: Rate-Distortion-Optimal Vector Quantization for Gaussian-Mixture Sourcescited by 1
  25. 2026MODE: Modality-Decomposed Expert-Level Mixed-Precision Quantization for MoE Multimodal LLMscited by 1
  26. 2026StagQ: Constraint-Driven Multi-Precision Weight Quantization for LLMscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Routing-Free Mixture-of-Experts
  2. 2025Opportunistic Expert Activation: Batch-Aware Expert Routing for Faster Decode Without Retraining
  3. 2026Quality-Constrained Routing over a Fixed Pool of Quantized Mixture-of-Experts Instances
  4. 2026Is Retraining-Free Enough? The Necessity of Router Calibration for Efficient MoE Compression
  5. 2026Share First, Route What Remains: A Unified Framework for Token-Adaptive MoE Computation
  6. 2026IntBMoE: Integrating Block-Level Conditioning into Expert Composition for Full-Participation Mixture-of-Experts
  7. 2026A Persistent State for Auditable Mixture-of-Experts Routing
  8. 2026DECO: Sparse Mixture-of-Experts with Dense-Comparable Performance on End-Side Devices