# PagedWeight: Efficient MoE LLM Serving with Dynamic Quality-Aware Weight Quantization

arXiv: https://arxiv.org/abs/2607.16184 · HTML: https://mlsanity.com/paper/2607.16184/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Efficient MoE training and inference systems, one of 12 directions in Mixture-of-experts.
- Builds on 30 of its 33 references that are in the map.
- First posted on arXiv 17 Jul 2026.

## Placed in

- Mixture-of-experts / [Efficient MoE training and inference systems](https://mlsanity.com/moe/efficient-moe-training-and-inference-systems/)
- Efficient LLM inference and serving / [Diffusion LLM and MoE inference](https://mlsanity.com/inference/diffusion-llm-and-moe-inference/)

## Builds on (most cited first, 30 of 30)

- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity](https://mlsanity.com/paper/2101.03961/) (2021)
- [GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers](https://mlsanity.com/paper/2210.17323/) (2022)
- [GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding](https://mlsanity.com/paper/2006.16668/) (2020)
- [SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models](https://mlsanity.com/paper/2211.10438/) (2022)
- [AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration](https://mlsanity.com/paper/2306.00978/) (2023)
- [FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPU](https://mlsanity.com/paper/2303.06865/) (2023)
- [SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compression](https://mlsanity.com/paper/2306.03078/) (2023)
- [Any-Precision LLM: Low-Cost Deployment of Multiple, Different-Sized LLMs](https://mlsanity.com/paper/2402.10517/) (2024)
- [KVTuner: Sensitivity-Aware Layer-Wise Mixed-Precision KV Cache Quantization for Efficient and Nearly Lossless LLM Inference](https://mlsanity.com/paper/2502.04420/) (2025)
- [Mixture of Quantized Experts (MoQE): Complementary Effect of Low-bit Quantization and Robustness](https://mlsanity.com/paper/2310.02410/) (2023)
- [MegaScale-Infer: Serving Mixture-of-Experts at Scale with Disaggregated Expert Parallelism](https://mlsanity.com/paper/2504.02263/) (2025)
- [MxMoE: Mixed-precision Quantization for MoE with Accuracy and Performance Co-Design](https://mlsanity.com/paper/2505.05799/) (2025)
- [DiffKV: Differentiated Memory Management for Large Language Models with Parallel KV Compaction](https://mlsanity.com/paper/2412.03131/) (2024)
- [Klotski: Efficient Mixture-of-Expert Inference via Expert-Aware Multi-Batch Pipeline](https://mlsanity.com/paper/2502.06888/) (2025)
- [ExpertFlow: Efficient Mixture-of-Experts Inference via Predictive Expert Caching and Token Scheduling](https://mlsanity.com/paper/2410.17954/) (2024)
- [AQUA: Network-Accelerated Memory Offloading for LLMs in Scale-Up GPU Domains](https://mlsanity.com/paper/2407.21255/) (2024)
- [MoQAE: Mixed-Precision Quantization for Long-Context LLM Inference via Mixture of Quantization-Aware Experts](https://mlsanity.com/paper/2506.07533/) (2025)
- [MoE-SpeQ: Speculative Quantized Decoding with Proactive Expert Prefetching and Offloading for Mixture-of-Experts](https://mlsanity.com/paper/2511.14102/) (2025)
- [EAQuant: Enhancing Post-Training Quantization for MoE Models via Expert-Aware Optimization](https://mlsanity.com/paper/2506.13329/) (2025)
- [QuantMoE-Bench: Examining Post-Training Quantization for Mixture-of-Experts](https://mlsanity.com/paper/2406.08155/) (2024)
- [MiLo: Efficient Quantized MoE Inference with Mixture of Low-Rank Compensators](https://mlsanity.com/paper/2504.02658/) (2025)
- [KVComp: A High-Performance, LLM-Aware, Lossy Compression Framework for KV Cache](https://mlsanity.com/paper/2509.00579/) (2025)
- [Dynamic Expert Quantization for Scalable Mixture-of-Experts Inference](https://mlsanity.com/paper/2511.15015/) (2025)
- [DP-LLM: Runtime Model Adaptation with Dynamic Layer-wise Precision Assignment](https://mlsanity.com/paper/2508.06041/) (2025)
- [MoE-Lens: Towards the Hardware Limit of High-Throughput MoE LLM Serving Under Resource Constraints](https://mlsanity.com/paper/2504.09345/) (2025)
- [DyMoE: Dynamic Expert Orchestration with Mixed-Precision Quantization for Efficient MoE Inference on Edge](https://mlsanity.com/paper/2603.19172/) (2026)
- [HybridGen: Efficient LLM Generative Inference via CPU-GPU Hybrid Computing](https://mlsanity.com/paper/2604.18529/) (2026)
- [ARQ: A Mixed-Precision Quantization Framework for Accurate and Certifiably Robust DNNs](https://mlsanity.com/paper/2410.24214/) (2024)
- [TTKV: Temporal-Tiered KV Cache for Long-Context LLM Inference](https://mlsanity.com/paper/2604.19769/) (2026)

## Built on by (most cited first, 2 of 2)

- [Every Expert Counts: ExactMoE for Memory-Efficient W4A16 Inference](https://mlsanity.com/paper/2608.15383/) (2026)
- [DuoRate: Two-Rate Per-Expert Precision Allocation for Mixture-of-Experts Serving](https://mlsanity.com/paper/2610.13328/) (2026)

## Abstract (arXiv)

Mixture-of-Experts (MoE) is a popular class of large language models (LLMs), offering high efficiency and accuracy. However, in KV-cache-intensive serving scenarios, MoEs often exhibit a tension between the GPU memory requirements of the model weights and the growing KV cache. We propose PagedWeight, a novel management method for MoE LLM serving that dynamically quantizes MoE model's weights at runtime and balances expert-weight precision with the KV cache sizes. PagedWeight exposes and effectively navigates the complex tradeoff between the model's task accuracy, memory consumption, and throughput/latency. Across several memory-sensitive MoE serving scenarios, PagedWeight improves the quality-memory tradeoff over several existing quantization baselines. PagedWeight achieves FP16-equivalent accuracy with up to 72.0% GPU memory savings and 1.94× throughput improvement, and improves quality over quantization methods by up to 39.3% at a similar memory budget with at most 4.1% throughput loss.