# KBVQ-MoE: KLT-guided SVD with Bias-Corrected Vector Quantization for MoE Large Language Models

arXiv: https://arxiv.org/abs/2602.11184 · HTML: https://mlsanity.com/paper/2602.11184/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Placed in MoE large language models, one of 12 directions in Mixture-of-experts.
- Builds on 27 of its 44 references that are in the map.
- First posted on arXiv 30 Jan 2026.

## Placed in

- Mixture-of-experts / [MoE large language models](https://mlsanity.com/moe/moe-large-language-models/)

## Builds on (most cited first, 27 of 27)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge](https://mlsanity.com/paper/1803.05457/) (2018)
- [HellaSwag: Can a Machine Really Finish Your Sentence?](https://mlsanity.com/paper/1905.07830/) (2019)
- [Pointer Sentinel Mixture Models](https://mlsanity.com/paper/1609.07843/) (2016)
- [GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers](https://mlsanity.com/paper/2210.17323/) (2022)
- [Mixtral of Experts](https://mlsanity.com/paper/2401.04088/) (2024)
- [DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model](https://mlsanity.com/paper/2405.04434/) (2024)
- [The LAMBADA dataset: Word prediction requiring a broad discourse context](https://mlsanity.com/paper/1606.06031/) (2016)
- [Kimi-VL Technical Report](https://mlsanity.com/paper/2504.07491/) (2025)
- [QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebooks](https://mlsanity.com/paper/2402.04396/) (2024)
- [Extreme Compression of Large Language Models via Additive Quantization](https://mlsanity.com/paper/2401.06118/) (2024)
- [FlatQuant: Flatness Matters for LLM Quantization](https://mlsanity.com/paper/2410.09426/) (2024)
- [OstQuant: Refining Large Language Model Quantization with Orthogonal and Scaling Transformations for Better Distribution Fitting](https://mlsanity.com/paper/2501.13987/) (2025)
- [GPTVQ: The Blessing of Dimensionality for LLM Quantization](https://mlsanity.com/paper/2402.15319/) (2024)
- [GPTAQ: Efficient Finetuning-Free Quantization for Asymmetric Calibration](https://mlsanity.com/paper/2504.02692/) (2025)
- [VPTQ: Extreme Low-bit Vector Post-Training Quantization for Large Language Models](https://mlsanity.com/paper/2409.17066/) (2024)
- [A Systematic Classification of Knowledge, Reasoning, and Context within the ARC Dataset](https://mlsanity.com/paper/1806.00358/) (2018)
- [MoEQuant: Enhancing Quantization for Mixture-of-Experts Large Language Models via Expert-Balanced Sampling and Affinity Guidance](https://mlsanity.com/paper/2505.03804/) (2025)
- [Delta Decompression for MoE-based LLMs Compression](https://mlsanity.com/paper/2502.17298/) (2025)
- [MambaQuant: Quantizing the Mamba Family with Variance Aligned Rotation Methods](https://mlsanity.com/paper/2501.13484/) (2025)
- [Sub-MoE: Efficient Mixture-of-Expert LLMs Compression via Subspace Expert Merging](https://mlsanity.com/paper/2506.23266/) (2025)
- [I-LLM: Efficient Integer-Only Inference for Fully-Quantized Low-Bit Large Language Models](https://mlsanity.com/paper/2405.17849/) (2024)
- [Load Balancing Mixture of Experts with Similarity Preserving Routers](https://mlsanity.com/paper/2506.14038/) (2025)
- [EAC-MoE: Expert-Selection Aware Compressor for Mixture-of-Experts Large Language Models](https://mlsanity.com/paper/2508.01625/) (2025)
- [SpecQuant: Spectral Decomposition and Adaptive Truncation for Ultra-Low-Bit LLMs Quantization](https://mlsanity.com/paper/2511.11663/) (2025)
- [PCDVQ: Enhancing Vector Quantization for Large Language Models via Polar Coordinate Decoupling](https://mlsanity.com/paper/2506.05432/) (2025)

## Built on by (most cited first, 8 of 8)

- [TWLA: Achieving Ternary Weights and Low-Bit Activations for LLMs via Post-Training Quantization](https://mlsanity.com/paper/2606.13054/) (2026)
- [BWLA: Breaking the Barrier of W1AX Post-Training Quantization for LLMs](https://mlsanity.com/paper/2605.00422/) (2026)
- [MoBiE: Efficient Inference of Mixture of Binary Experts under Post-Training Quantization](https://mlsanity.com/paper/2604.06798/) (2026)
- [Rubric-Guided Process Reward for Stepwise Model Routing](https://mlsanity.com/paper/2605.29310/) (2026)
- [All for 1-Bit: Towards Genuine 1-Bit Post-Training Quantization for LLMs](https://mlsanity.com/paper/2609.06161/) (2026)
- [JustQuant: You Don't Need Smoothing, SVD, or Rotation for 4-Bit Activation Quantization](https://mlsanity.com/paper/2609.33601/) (2026)
- [ITC-MoE: Importance-guided Token-aware Compression for MoE Diffusion Language Models](https://mlsanity.com/paper/2610.01296/) (2026)
- [LoopQuant: Accurate and Efficient Post-Training Quantization for Looped Transformers](https://mlsanity.com/paper/2610.16368/) (2026)

## Abstract (arXiv)

Mixture of Experts (MoE) models have achieved great success by significantly improving performance while maintaining computational efficiency through sparse expert activation. However, their enormous parameter sizes and memory demands pose major challenges for deployment in resource-constrained environments. Vector Quantization (VQ) offers a promising approach for ultra-low-bit compression in Large Language Models (LLMs) by leveraging a codebook, where weight vectors are mapped to the most similar discrete codewords. Yet, directly applying VQ to MoEs often leads to substantial performance degradation due to two critical obstacles: (1) redundant representations among experts cause VQ to repeatedly quantize similar representations for each expert, resulting in inefficient use of limited codebook capacity; and (2) cumulative output bias is amplified by expert aggregation in MoE layers, leading to distributional shifts in the quantized outputs. To address these issues, we propose KBVQ-MoE, a novel VQ framework to enhance extremely low-bit quantization for MoE-based LLMs. KBVQ-MoE integrates two techniques: (1) input-driven redundancy elimination, where a Karhunen-Loeve Transform (KLT) guided singular value decomposition (SVD) extracts dominant weight components and shares them across experts; and (2) bias-corrected output stabilization, where vector quantization is applied only to expert-specific (non-redundant) representations and the quantized outputs are corrected via channel-wise affine compensation. Experiments on various MoE LLMs demonstrate that KBVQ-MoE preserves accuracy substantially better than existing quantization methods. For example, 3-bit quantization of Qwen1.5-MoE-A2.7B achieves an average accuracy of 67.99, nearly identical to the FP16 baseline of 68.07, underscoring KBVQ-MoE's potential for efficient deployment on edge devices and other resource-constrained platforms.