mlsanityData to 14 Oct 2026

BitsMoE: Efficient Spectral Energy-Guided Bit Allocation for MoE LLM Quantization

arXiv 2606.00079 ↗First posted 22 May 2026cs.LG cs.AIcited by 0

On the map: Mixture-of-experts → MoE large language models

Abstract (arXiv)

Mixture-of-Experts (MoE) large language models incur substantial memory costs due to their large expert parameter counts. Mixed-precision quantization reduces these costs by allocating different bit-widths to experts or linear blocks according to their importance. However, assigning a single precision within each expert or linear block overlooks its internal structural heterogeneity. This limitation motivates two key questions: (1) how to define a fine-grained unit for quantization within a linear transformation; and (2) how to characterize the quantization cost of each unit under actual activation patterns and different bit-widths. To address these two questions, we propose BitsMoE, a cost-aware mixed-precision quantization framework built on two complementary techniques: (1) Shared-basis Spectral Decomposition (SSD) separates expert weights into a shared basis and expert-specific spectral components, defining structural quantization units while exploiting cross-expert redundancy. (2) Factorized Quantization Cost Modeling (FQCM) estimates component-wise costs from output reconstruction loss by combining intrinsic spectral importance, activation-dependent importance, and bit-width-dependent distortion. Using these component-wise costs, we formulate bit allocation as an integer linear program (ILP) that minimizes total modeled quantization cost under a fixed memory budget. On Qwen3-30B-A3B at 2-bit, BitsMoE achieves 64.29% average accuracy over seven downstream tasks, outperforming the evaluated MoE-specific methods, including those using ILP-based bit allocation, and exceeding GEMQ by 2.80 percentage points. Under the same setting, it achieves a 16.47× end-to-end offline quantization speedup over GEMQ. It also achieves up to 6.46× the decode throughput of GPTQ.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in MoE large language models, one of 12 directions in Mixture-of-experts. Builds on 36 of its 45 references that are in the map. First posted on arXiv 22 May 2026.

Builds on

36 of 45 references.

Closest ancestors · 14

Same direction, newest first

  1. 2026GEMQ: Global Expert-Level Mixed-Precision Quantization for MoE LLMs21 May
  2. 2025Qwen3 Technical Report14 May
  3. 2025MxMoE: Mixed-precision Quantization for MoE with Accuracy and Performance Co-Design9 May
  4. 2025MoEQuant: Enhancing Quantization for Mixture-of-Experts Large Language Models via Expert-Balanced Sampling and Affinity Guidance2 May
  5. 2025Delta Decompression for MoE-based LLMs Compression24 Feb
  6. 2024MoE-I2: Compressing Mixture of Experts Models through Inter-Expert Pruning and Intra-Expert Low-Rank Decomposition1 Nov
  7. 2024OLMoE: Open Mixture-of-Experts Language Models3 Sep
  8. 2024Efficient Expert Pruning for Sparse Mixture-of-Experts Language Models: Enhancing Performance and Reducing Inference Costs1 Jul
  9. 2024A Survey on Mixture of Experts in Large Language Models26 Jun
  10. 2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model7 May
  11. 2024Not All Experts are Equal: Efficient Expert Pruning and Skipping for Mixture-of-Experts Large Language Models22 Feb
  12. 2024OpenMoE: An Early Effort on Open Mixture-of-Experts Language Models29 Jan
2 more
  1. 2024Mixtral of Experts8 Jan
  2. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer23 Jan

Other references · 22

  1. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  2. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  3. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  4. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  5. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  6. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,100
  7. 2016Pointer Sentinel Mixture Modelscited by 3,096
  8. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,073
  9. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,164
  10. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
  11. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
  12. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,298
10 more
  1. 2019MathQA: Towards Interpretable Math Word Problem Solving with Operation-Based Formalismscited by 662
  2. 2024QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMscited by 634
  3. 2023OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Modelscited by 424
  4. 2024SVD-LLM: Truncation-aware Singular Value Decomposition for Large Language Model Compressioncited by 243
  5. 2022Language model compression with weighted low-rank factorizationcited by 195
  6. 2023ASVD: Activation-aware Singular Value Decomposition for Compressing Large Language Modelscited by 169
  7. 2024MARLIN: Mixed-Precision Auto-Regressive Parallel Inference on Large Language Modelscited by 52
  8. 2024Delta-CoMe: Training-Free Delta-Compression with Mixed-Precision for Large Language Modelscited by 17
  9. 2022PAD-Net: An Efficient Framework for Dynamic Networkscited by 9
  10. 2025MiLo: Efficient Quantized MoE Inference with Mixture of Low-Rank Compensatorscited by 8

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Mixture Compressor for Mixture-of-Experts LLMs Gains More
  2. 2026KBVQ-MoE: KLT-guided SVD with Bias-Corrected Vector Quantization for MoE Large Language Models
  3. 2024QuantMoE-Bench: Examining Post-Training Quantization for Mixture-of-Experts
  4. 2026Q-Strata: Hierarchical Bit Allocation for Mixed-Precision Quantization of Mixture-of-Experts LLMs
  5. 2026MonoMoE: An Efficient Fused Mega-kernel for Quantized MoE Decoding
  6. 2025CAMERA: Multi-Matrix Joint Compression for MoE Models via Micro-Expert Redundancy Analysis
  7. 2025MC#: Mixture Compressor for Mixture-of-Experts Large Models
  8. 2026Beyond Expert Rankings: Contrast Calibration for Mixed-Precision MoE Quantization