# MxMoE: Mixed-precision Quantization for MoE with Accuracy and Performance Co-Design

arXiv: https://arxiv.org/abs/2505.05799 · HTML: https://mlsanity.com/paper/2505.05799/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Placed in MoE large language models, one of 12 directions in Mixture-of-experts.
- Builds on 26 of its 33 references that are in the map.
- First posted on arXiv 9 May 2025.

## Placed in

- Mixture-of-experts / [MoE large language models](https://mlsanity.com/moe/moe-large-language-models/)

## Builds on (most cited first, 26 of 26)

- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [Pointer Sentinel Mixture Models](https://mlsanity.com/paper/1609.07843/) (2016)
- [GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers](https://mlsanity.com/paper/2210.17323/) (2022)
- [Mixtral of Experts](https://mlsanity.com/paper/2401.04088/) (2024)
- [SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models](https://mlsanity.com/paper/2211.10438/) (2022)
- [AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration](https://mlsanity.com/paper/2306.00978/) (2023)
- [DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model](https://mlsanity.com/paper/2405.04434/) (2024)
- [KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cache](https://mlsanity.com/paper/2402.02750/) (2024)
- [QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMs](https://mlsanity.com/paper/2404.00456/) (2024)
- [KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization](https://mlsanity.com/paper/2401.18079/) (2024)
- [HAQ: Hardware-Aware Automated Quantization with Mixed Precision](https://mlsanity.com/paper/1811.08886/) (2018)
- [HAWQ: Hessian AWare Quantization of Neural Networks with Mixed-Precision](https://mlsanity.com/paper/1905.03696/) (2019)
- [OLMoE: Open Mixture-of-Experts Language Models](https://mlsanity.com/paper/2409.02060/) (2024)
- [CodeGeeX: A Pre-Trained Model for Code Generation with Multilingual Benchmarking on HumanEval-X](https://mlsanity.com/paper/2303.17568/) (2023)
- [Massive Activations in Large Language Models](https://mlsanity.com/paper/2402.17762/) (2024)
- [SqueezeLLM: Dense-and-Sparse Quantization](https://mlsanity.com/paper/2306.07629/) (2023)
- [Atom: Low-bit Quantization for Efficient and Accurate LLM Serving](https://mlsanity.com/paper/2310.19102/) (2023)
- [QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Serving](https://mlsanity.com/paper/2405.04532/) (2024)
- [Low-bit Quantization of Neural Networks for Efficient Inference](https://mlsanity.com/paper/1902.06822/) (2019)
- [OpenMoE: An Early Effort on Open Mixture-of-Experts Language Models](https://mlsanity.com/paper/2402.01739/) (2024)
- [WKVQuant: Quantizing Weight and Key/Value Cache for Large Language Models Gains More](https://mlsanity.com/paper/2402.12065/) (2024)
- [HOBBIT: A Mixed Precision Expert Offloading System for Fast MoE Inference](https://mlsanity.com/paper/2411.01433/) (2024)
- [SKVQ: Sliding-window Key and Value Cache Quantization for Large Language Models](https://mlsanity.com/paper/2405.06219/) (2024)
- [Mixture Compressor for Mixture-of-Experts LLMs Gains More](https://mlsanity.com/paper/2410.06270/) (2024)
- [Who Says Elephants Can't Run: Bringing Large Scale MoE Models into Cloud Scale Production](https://mlsanity.com/paper/2211.10017/) (2022)

## Built on by (most cited first, 30 of 32)

- [Get Experience from Practice: LLM Agents with Record & Replay](https://mlsanity.com/paper/2505.17716/) (2025)
- [Unveiling Super Experts in Mixture-of-Experts Large Language Models](https://mlsanity.com/paper/2507.23279/) (2025)
- [MoDES: Accelerating Mixture-of-Experts Multimodal Large Language Models via Dynamic Expert Skipping](https://mlsanity.com/paper/2511.15690/) (2025)
- [PuzzleMoE: Efficient Compression of Large Mixture-of-Experts Models via Sparse Expert Merging and Bit-packed inference](https://mlsanity.com/paper/2511.04805/) (2025)
- [LayerScope: Predictive Cross-Layer Scheduling for Efficient Multi-Batch MoE Inference on Legacy Servers](https://mlsanity.com/paper/2509.23638/) (2025)
- [Dynamic Expert Quantization for Scalable Mixture-of-Experts Inference](https://mlsanity.com/paper/2511.15015/) (2025)
- [GEMQ: Global Expert-Level Mixed-Precision Quantization for MoE LLMs](https://mlsanity.com/paper/2605.23078/) (2026)
- [AlphaQ: Calibration-Free Bit Allocation for Mixture-of-Experts Quantization](https://mlsanity.com/paper/2606.04980/) (2026)
- [ZipMoE: Efficient On-Device MoE Serving via Lossless Compression and Cache-Affinity Scheduling](https://mlsanity.com/paper/2601.21198/) (2026)
- [VEQ: Modality-Adaptive Quantization for MoE Vision-Language Models](https://mlsanity.com/paper/2602.01037/) (2026)
- [Bandwidth-Efficient Adaptive Mixture-of-Experts via Low-Rank Compensation](https://mlsanity.com/paper/2512.17073/) (2025)
- [Model Compression with Exact Budget Constraints via Riemannian Manifolds](https://mlsanity.com/paper/2605.00649/) (2026)
- [Q-Strata: Hierarchical Bit Allocation for Mixed-Precision Quantization of Mixture-of-Experts LLMs](https://mlsanity.com/paper/2608.30564/) (2026)
- [PagedWeight: Efficient MoE LLM Serving with Dynamic Quality-Aware Weight Quantization](https://mlsanity.com/paper/2607.16184/) (2026)
- [MoE-Compression: How the Compression Error of Experts Affects the Inference Accuracy of MoE Model?](https://mlsanity.com/paper/2509.07727/) (2025)
- [SignRoundV2: Toward Closing the Performance Gap in Extremely Low-Bit Post-Training Quantization for LLMs](https://mlsanity.com/paper/2512.04746/) (2025)
- [CoGR-MoE: Concept-Guided Expert Routing with Consistent Selection and Flexible Reasoning for Visual Question Answering](https://mlsanity.com/paper/2604.16930/) (2026)
- [TileQ: Efficient Low-Rank Quantization of Mixture-of-Experts with 2D Tiling](https://mlsanity.com/paper/2605.09281/) (2026)
- [MODE: Modality-Decomposed Expert-Level Mixed-Precision Quantization for MoE Multimodal LLMs](https://mlsanity.com/paper/2606.17118/) (2026)
- [VisMMOE: Exploiting Visual-Expert Affinity for Efficient Visual-Language MoE Offloading](https://mlsanity.com/paper/2605.05899/) (2026)
- [BitsMoE: Efficient Spectral Energy-Guided Bit Allocation for MoE LLM Quantization](https://mlsanity.com/paper/2606.00079/) (2026)
- [Agentic evolution of physically constrained foundation models](https://mlsanity.com/paper/2606.25532/) (2026)
- [Every Expert Counts: ExactMoE for Memory-Efficient W4A16 Inference](https://mlsanity.com/paper/2608.15383/) (2026)
- [Colla-Q: Toward Collaborative Experts in MoE Quantization via Minimax Precision Balancing](https://mlsanity.com/paper/2609.18131/) (2026)
- [ITC-MoE: Importance-guided Token-aware Compression for MoE Diffusion Language Models](https://mlsanity.com/paper/2610.01296/) (2026)
- [AlignQuant: Tile-Aligned Mixed-Precision Quantization for Efficient LLM Generation](https://mlsanity.com/paper/2610.07457/) (2026)
- [DuoRate: Two-Rate Per-Expert Precision Allocation for Mixture-of-Experts Serving](https://mlsanity.com/paper/2610.13328/) (2026)
- [The Price of Precision: A Calibrated Cost Model for Mixed-Precision LLM Serving](https://mlsanity.com/paper/2610.15629/) (2026)
- [Context-Aware Precision Scheduling for Small Language Models](https://mlsanity.com/paper/2610.15640/) (2026)
- [Beyond Expert Rankings: Contrast Calibration for Mixed-Precision MoE Quantization](https://mlsanity.com/paper/2610.20353/) (2026)

## Abstract (arXiv)

Mixture-of-Experts (MoE) models face deployment challenges due to their large parameter counts and computational demands. We explore quantization for MoE models and highlight two key insights: 1) linear blocks exhibit varying quantization sensitivity, and 2) divergent expert activation frequencies create heterogeneous computational characteristics. Based on these observations, we introduce MxMoE, a mixed-precision optimization framework for MoE models that considers both algorithmic and system perspectives. MxMoE navigates the design space defined by parameter sensitivity, expert activation dynamics, and hardware resources to derive efficient mixed-precision configurations. Additionally, MxMoE automatically generates optimized mixed-precision GroupGEMM kernels, enabling parallel execution of GEMMs with different precisions. Evaluations show that MxMoE outperforms existing methods, achieving 2.4 lower Wikitext-2 perplexity than GPTQ at 2.25-bit and delivering up to 3.4x speedup over full precision, as well as up to 29.4% speedup over uniform quantization at equivalent accuracy with 5-bit weight-activation quantization. Our code is available at https://github.com/cat538/MxMoE.