mlsanityData to 7 Oct 2026

PagedWeight: Efficient MoE LLM Serving with Dynamic Quality-Aware Weight Quantization

arXiv 2607.16184 ↗First posted 17 Jul 2026cs.LGcited by 1

On the map: Mixture-of-experts → Efficient MoE training and inference systems

also in 1 more
Abstract (arXiv)

Mixture-of-Experts (MoE) is a popular class of large language models (LLMs), offering high efficiency and accuracy. However, in KV-cache-intensive serving scenarios, MoEs often exhibit a tension between the GPU memory requirements of the model weights and the growing KV cache. We propose PagedWeight, a novel management method for MoE LLM serving that dynamically quantizes MoE model's weights at runtime and balances expert-weight precision with the KV cache sizes. PagedWeight exposes and effectively navigates the complex tradeoff between the model's task accuracy, memory consumption, and throughput/latency. Across several memory-sensitive MoE serving scenarios, PagedWeight improves the quality-memory tradeoff over several existing quantization baselines. PagedWeight achieves FP16-equivalent accuracy with up to 72.0% GPU memory savings and 1.94× throughput improvement, and improves quality over quantization methods by up to 39.3% at a similar memory budget with at most 4.1% throughput loss.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Efficient MoE training and inference systems, one of 12 directions in Mixture-of-experts. Builds on 30 of its 33 references that are in the map. First posted on arXiv 17 Jul 2026.

Builds on

30 of 33 references.

Closest ancestors · 8

Same direction, newest first

  1. 2026DyMoE: Dynamic Expert Orchestration with Mixed-Precision Quantization for Efficient MoE Inference on Edge19 Mar
  2. 2025Dynamic Expert Quantization for Scalable Mixture-of-Experts Inference19 Nov
  3. 2025MoE-SpeQ: Speculative Quantized Decoding with Proactive Expert Prefetching and Offloading for Mixture-of-Experts18 Nov
  4. 2025MoE-Lens: Towards the Hardware Limit of High-Throughput MoE LLM Serving Under Resource Constraints12 Apr
  5. 2025MiLo: Efficient Quantized MoE Inference with Mixture of Low-Rank Compensators3 Apr
  6. 2025MegaScale-Infer: Serving Mixture-of-Experts at Scale with Disaggregated Expert Parallelism3 Apr
  7. 2025Klotski: Efficient Mixture-of-Expert Inference via Expert-Aware Multi-Batch Pipeline9 Feb
  8. 2024ExpertFlow: Efficient Mixture-of-Experts Inference via Predictive Expert Caching and Token Scheduling23 Oct

Other references · 22

  1. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  2. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
  3. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,838
  4. 2020GShard: Scaling Giant Models with Conditional Computation and Automatic Shardingcited by 1,601
  5. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,338
  6. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,234
  7. 2023FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPUcited by 567
  8. 2023SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compressioncited by 359
  9. 2024Any-Precision LLM: Low-Cost Deployment of Multiple, Different-Sized LLMscited by 50
  10. 2025KVTuner: Sensitivity-Aware Layer-Wise Mixed-Precision KV Cache Quantization for Efficient and Nearly Lossless LLM Inferencecited by 36
  11. 2023Mixture of Quantized Experts (MoQE): Complementary Effect of Low-bit Quantization and Robustnesscited by 34
  12. 2024DiffKV: Differentiated Memory Management for Large Language Models with Parallel KV Compactioncited by 26
10 more
  1. 2025MxMoE: Mixed-precision Quantization for MoE with Accuracy and Performance Co-Designcited by 26
  2. 2024AQUA: Network-Accelerated Memory Offloading for LLMs in Scale-Up GPU Domainscited by 14
  3. 2025MoQAE: Mixed-Precision Quantization for Long-Context LLM Inference via Mixture of Quantization-Aware Expertscited by 12
  4. 2024QuantMoE-Bench: Examining Post-Training Quantization for Mixture-of-Expertscited by 9
  5. 2025EAQuant: Enhancing Post-Training Quantization for MoE Models via Expert-Aware Optimizationcited by 9
  6. 2025KVComp: A High-Performance, LLM-Aware, Lossy Compression Framework for KV Cachecited by 6
  7. 2025DP-LLM: Runtime Model Adaptation with Dynamic Layer-wise Precision Assignmentcited by 4
  8. 2026HybridGen: Efficient LLM Generative Inference via CPU-GPU Hybrid Computingcited by 2
  9. 2024ARQ: A Mixed-Precision Quantization Framework for Accurate and Certifiably Robust DNNscited by 1
  10. 2026TTKV: Temporal-Tiered KV Cache for Long-Context LLM Inferencecited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Every Expert Counts: ExactMoE for Memory-Efficient W4A16 Inferencecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Faster MoE LLM Inference for Extremely Large Models
  2. 2026GEMQ: Global Expert-Level Mixed-Precision Quantization for MoE LLMs
  3. 2025Grove MoE: Towards Efficient and Superior MoE LLMs with Adjugate Experts
  4. 2025A3D-MoE: Acceleration of Large Language Models with Mixture of Experts via 3D Heterogeneous Integration
  5. 2025D2MoE: Dual Routing and Dynamic Scheduling for Efficient On-Device MoE-based LLM Serving
  6. 2024Understanding the Performance and Estimating the Cost of LLM Fine-Tuning
  7. 2026Benchmarking Composable Compression Techniques in Mixture-of-Experts LLMs
  8. 2025Taming Latency-Memory Trade-Off in MoE-Based LLM Serving via Fine-Grained Expert Offloading