mlsanityData to 14 Oct 2026

MxMoE: Mixed-precision Quantization for MoE with Accuracy and Performance Co-Design

arXiv 2505.05799 ↗First posted 9 May 2025cs.LG cs.AIcited by 30

On the map: Mixture-of-experts → MoE large language models

Abstract (arXiv)

Mixture-of-Experts (MoE) models face deployment challenges due to their large parameter counts and computational demands. We explore quantization for MoE models and highlight two key insights: 1) linear blocks exhibit varying quantization sensitivity, and 2) divergent expert activation frequencies create heterogeneous computational characteristics. Based on these observations, we introduce MxMoE, a mixed-precision optimization framework for MoE models that considers both algorithmic and system perspectives. MxMoE navigates the design space defined by parameter sensitivity, expert activation dynamics, and hardware resources to derive efficient mixed-precision configurations. Additionally, MxMoE automatically generates optimized mixed-precision GroupGEMM kernels, enabling parallel execution of GEMMs with different precisions. Evaluations show that MxMoE outperforms existing methods, achieving 2.4 lower Wikitext-2 perplexity than GPTQ at 2.25-bit and delivering up to 3.4x speedup over full precision, as well as up to 29.4% speedup over uniform quantization at equivalent accuracy with 5-bit weight-activation quantization. Our code is available at https://github.com/cat538/MxMoE.

Key facts. Cited by 30 papers in the map (data through 14 Oct 2026). Placed in MoE large language models, one of 12 directions in Mixture-of-experts. Builds on 26 of its 33 references that are in the map. First posted on arXiv 9 May 2025.

Builds on

26 of 33 references.

Closest ancestors · 7

Same direction, newest first

  1. 2024Qwen2.5 Technical Report19 Dec
  2. 2024Mixture Compressor for Mixture-of-Experts LLMs Gains More8 Oct
  3. 2024OLMoE: Open Mixture-of-Experts Language Models3 Sep
  4. 2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model7 May
  5. 2024OpenMoE: An Early Effort on Open Mixture-of-Experts Language Models29 Jan
  6. 2024Mixtral of Experts8 Jan
  7. 2022Who Says Elephants Can't Run: Bringing Large Scale MoE Models into Cloud Scale Production18 Nov

Other references · 19

  1. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  2. 2016Pointer Sentinel Mixture Modelscited by 3,096
  3. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
  4. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
  5. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,298
  6. 2024KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cachecited by 637
  7. 2024QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMscited by 634
  8. 2024KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantizationcited by 591
  9. 2018HAQ: Hardware-Aware Automated Quantization with Mixed Precisioncited by 443
  10. 2019HAWQ: Hessian AWare Quantization of Neural Networks with Mixed-Precisioncited by 402
  11. 2023CodeGeeX: A Pre-Trained Model for Code Generation with Multilingual Benchmarking on HumanEval-Xcited by 300
  12. 2023SqueezeLLM: Dense-and-Sparse Quantizationcited by 294
7 more
  1. 2024Massive Activations in Large Language Modelscited by 294
  2. 2023Atom: Low-bit Quantization for Efficient and Accurate LLM Servingcited by 272
  3. 2024QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Servingcited by 208
  4. 2019Low-bit Quantization of Neural Networks for Efficient Inferencecited by 193
  5. 2024WKVQuant: Quantizing Weight and Key/Value Cache for Large Language Models Gains Morecited by 69
  6. 2024HOBBIT: A Mixed Precision Expert Offloading System for Fast MoE Inferencecited by 55
  7. 2024SKVQ: Sliding-window Key and Value Cache Quantization for Large Language Modelscited by 44

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 9; 2026: 21.

30 papers.

Sort:

By area:

  1. 2025Get Experience from Practice: LLM Agents with Record & Replaycited by 23
  2. 2025Unveiling Super Experts in Mixture-of-Experts Large Language Modelscited by 17
  3. 2025MoDES: Accelerating Mixture-of-Experts Multimodal Large Language Models via Dynamic Expert Skippingcited by 11
  4. 2025PuzzleMoE: Efficient Compression of Large Mixture-of-Experts Models via Sparse Expert Merging and Bit-packed inferencecited by 9
  5. 2025LayerScope: Predictive Cross-Layer Scheduling for Efficient Multi-Batch MoE Inference on Legacy Serverscited by 8
  6. 2025Dynamic Expert Quantization for Scalable Mixture-of-Experts Inferencecited by 7
  7. 2026GEMQ: Global Expert-Level Mixed-Precision Quantization for MoE LLMscited by 7
  8. 2026AlphaQ: Calibration-Free Bit Allocation for Mixture-of-Experts Quantizationcited by 7
  9. 2026ZipMoE: Efficient On-Device MoE Serving via Lossless Compression and Cache-Affinity Schedulingcited by 4
  10. 2026VEQ: Modality-Adaptive Quantization for MoE Vision-Language Modelscited by 4
  11. 2025Bandwidth-Efficient Adaptive Mixture-of-Experts via Low-Rank Compensationcited by 3
  12. 2026Model Compression with Exact Budget Constraints via Riemannian Manifoldscited by 3
18 more
  1. 2026Q-Strata: Hierarchical Bit Allocation for Mixed-Precision Quantization of Mixture-of-Experts LLMscited by 3
  2. 2026PagedWeight: Efficient MoE LLM Serving with Dynamic Quality-Aware Weight Quantizationcited by 2
  3. 2025MoE-Compression: How the Compression Error of Experts Affects the Inference Accuracy of MoE Model?cited by 1
  4. 2026CoGR-MoE: Concept-Guided Expert Routing with Consistent Selection and Flexible Reasoning for Visual Question Answeringcited by 1
  5. 2026TileQ: Efficient Low-Rank Quantization of Mixture-of-Experts with 2D Tilingcited by 1
  6. 2026MODE: Modality-Decomposed Expert-Level Mixed-Precision Quantization for MoE Multimodal LLMscited by 1
  7. 2025SignRoundV2: Toward Closing the Performance Gap in Extremely Low-Bit Post-Training Quantization for LLMscited by 0
  8. 2026VisMMOE: Exploiting Visual-Expert Affinity for Efficient Visual-Language MoE Offloadingcited by 0
  9. 2026BitsMoE: Efficient Spectral Energy-Guided Bit Allocation for MoE LLM Quantizationcited by 0
  10. 2026Agentic evolution of physically constrained foundation modelscited by 0
  11. 2026Every Expert Counts: ExactMoE for Memory-Efficient W4A16 Inferencecited by 0
  12. 2026Colla-Q: Toward Collaborative Experts in MoE Quantization via Minimax Precision Balancingcited by 0
  13. 2026ITC-MoE: Importance-guided Token-aware Compression for MoE Diffusion Language Modelscited by 0
  14. 2026AlignQuant: Tile-Aligned Mixed-Precision Quantization for Efficient LLM Generationcited by 0
  15. 2026DuoRate: Two-Rate Per-Expert Precision Allocation for Mixture-of-Experts Servingcited by 0
  16. 2026The Price of Precision: A Calibrated Cost Model for Mixed-Precision LLM Servingcited by 0
  17. 2026Context-Aware Precision Scheduling for Small Language Modelscited by 0
  18. 2026Beyond Expert Rankings: Contrast Calibration for Mixed-Precision MoE Quantizationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026LatentMoE: Toward Optimal Accuracy per FLOP and Parameter in Mixture of Experts
  2. 2026MonoMoE: An Efficient Fused Mega-kernel for Quantized MoE Decoding
  3. 2026Hardware-Native Joint Sparse-Quantization for Trillion-Scale Mixture-of-Experts
  4. 2025MiLo: Efficient Quantized MoE Inference with Mixture of Low-Rank Compensators
  5. 2025MoPEQ: Mixture of Mixed Precision Quantized Experts
  6. 2024QuantMoE-Bench: Examining Post-Training Quantization for Mixture-of-Experts
  7. 2025MC#: Mixture Compressor for Mixture-of-Experts Large Models
  8. 2025MoE-CAP: Benchmarking Cost, Accuracy and Performance of Sparse Mixture-of-Experts Systems