mlsanityData to 14 Oct 2026

GEMQ: Global Expert-Level Mixed-Precision Quantization for MoE LLMs

arXiv 2605.23078 ↗First posted 21 May 2026cs.LG cs.CLcited by 7

On the map: Mixture-of-experts → MoE large language models

Abstract (arXiv)

Mixture-of-Experts Large Language Models (MoE-LLMs) achieve strong performance but incur substantial memory overhead due to massive expert parameters. Mixed-precision quantization mitigates this cost by allocating expert-wise bit-widths based on their importance, approaching the accuracy-memory Pareto frontier and enabling extreme low-bit quantization. However, existing methods rely on layer-wise importance estimation and overlook router shifts induced by quantization, resulting in suboptimal allocation and routing. In this work, we propose Global Expert-level Mixed-precision Quantization (GEMQ) to overcome these limitations via (1) a global linear-programming formulation that captures model-wide expert importance based on quantization error analysis, and (2) efficient router fine-tuning to adapt routing to quantized experts. These components are integrated into a progressive quantization framework that iteratively refines importance estimation and allocation. Experiments demonstrate that GEMQ significantly reduces memory and accelerates inference with minimal accuracy degradation. Source code is available at https://github.com/jndeng/GEMQ .

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Placed in MoE large language models, one of 12 directions in Mixture-of-experts. Builds on 44 of its 52 references that are in the map. First posted on arXiv 21 May 2026.

Builds on

44 of 52 references.

Closest ancestors · 15

Same direction, newest first

  1. 2025EAC-MoE: Expert-Selection Aware Compressor for Mixture-of-Experts Large Language Models3 Aug
  2. 2025EAQuant: Enhancing Post-Training Quantization for MoE Models via Expert-Aware Optimization16 Jun
  3. 2025Qwen3 Technical Report14 May
  4. 2025MxMoE: Mixed-precision Quantization for MoE with Accuracy and Performance Co-Design9 May
  5. 2025MoEQuant: Enhancing Quantization for Mixture-of-Experts Large Language Models via Expert-Balanced Sampling and Affinity Guidance2 May
  6. 2025DynaMo: Runtime Switchable Quantization for MoE with Cross-Dataset Adaptation27 Mar
  7. 2024Mixture Compressor for Mixture-of-Experts LLMs Gains More8 Oct
  8. 2024OLMoE: Open Mixture-of-Experts Language Models3 Sep
  9. 2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model7 May
  10. 2024Not All Experts are Equal: Efficient Expert Pruning and Skipping for Mixture-of-Experts Large Language Models22 Feb
  11. 2024DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models11 Jan
  12. 2024Mixtral of Experts8 Jan
3 more
  1. 2023QMoE: Practical Sub-1-Bit Compression of Trillion-Parameter Models25 Oct
  2. 2023Mixture of Quantized Experts (MoQE): Complementary Effect of Low-bit Quantization and Robustness3 Oct
  3. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer23 Jan

Other references · 29

  1. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  2. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  3. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  4. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  5. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  6. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,100
  7. 2016Pointer Sentinel Mixture Modelscited by 3,096
  8. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,883
  9. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
  10. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
  11. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,298
  12. 2024QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMscited by 634
17 more
  1. 2020Up or Down? Adaptive Rounding for Post-Training Quantizationcited by 621
  2. 2024Large Language Models: A Surveycited by 525
  3. 2021BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstructioncited by 479
  4. 2023OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Modelscited by 424
  5. 2024QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebookscited by 378
  6. 2023LLM-QAT: Data-Free Quantization Aware Training for Large Language Modelscited by 324
  7. 2023SqueezeLLM: Dense-and-Sparse Quantizationcited by 294
  8. 2019HAWQ-V2: Hessian Aware trace-Weighted Quantization of Neural Networkscited by 261
  9. 2022MegaBlocks: Efficient Sparse Training with Mixture-of-Expertscited by 209
  10. 2023Efficient Large Language Models: A Surveycited by 186
  11. 2024BiLLM: Pushing the Limit of Post-Training Quantization for LLMscited by 144
  12. 2017Practical Gauss-Newton Optimisation for Deep Learningcited by 142
  13. 2024FlatQuant: Flatness Matters for LLM Quantizationcited by 131
  14. 2024EfficientQAT: Efficient Quantization-Aware Training for Large Language Modelscited by 127
  15. 2024SliM-LLM: Salience-Driven Mixed-Precision Quantization for Large Language Modelscited by 63
  16. 2025ParetoQ: Improving Scaling Laws in Extremely Low-bit LLM Quantizationcited by 46
  17. 2025Preserving LLM Capabilities through Calibration Data Curation: From Analysis to Optimizationcited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 7.

7 papers.

  1. 2026Q-Strata: Hierarchical Bit Allocation for Mixed-Precision Quantization of Mixture-of-Experts LLMscited by 3
  2. 2026Detecting a Route Flip Is Easier Than Knowing Whether to Fix It: Causal Route-Mediated Damage in Quantized Mixture-of-Expertscited by 2
  3. 2026Beyond Activation Alignment:The Alignment-Diversity Tradeoff in Task-Aware LLM Quantizationcited by 1
  4. 2026When Can Depth Replace Precision? A Resource Theory of Quantized Neural Computationcited by 1
  5. 2026BitsMoE: Efficient Spectral Energy-Guided Bit Allocation for MoE LLM Quantizationcited by 0
  6. 2026Pricing the Risk of Runtime Compression: Anytime-Valid Admission and a Served-Output Law for Compressed Serving Statecited by 0
  7. 2026DuoRate: Two-Rate Per-Expert Precision Allocation for Mixture-of-Experts Servingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026AlphaQ: Calibration-Free Bit Allocation for Mixture-of-Experts Quantization
  2. 2025MC#: Mixture Compressor for Mixture-of-Experts Large Models
  3. 2025MoQAE: Mixed-Precision Quantization for Long-Context LLM Inference via Mixture of Quantization-Aware Experts
  4. 2026Beyond Expert Rankings: Contrast Calibration for Mixed-Precision MoE Quantization
  5. 2024Channel-Wise Mixed-Precision Quantization for Large Language Models
  6. 2025MoPEQ: Mixture of Mixed Precision Quantized Experts
  7. 2024APTQ: Attention-aware Post-Training Mixed-Precision Quantization for Large Language Models
  8. 2026Efficient Quantization of Mixture-of-Experts with Theoretical Generalization Guarantees