mlsanityData to 14 Oct 2026

UniSVQ: 2-bit Unified Scalar-Vector Quantization

arXiv 2606.10520 ↗First posted 9 Jun 2026cs.CLcited by 2

On the map: Efficient LLM inference and serving → KV cache quantization

Abstract (arXiv)

Post-training quantization at the 2-bit level enables low-cost deployment and inference acceleration for large language models (LLMs). Scalar quantization (SQ) and vector quantization (VQ) are two primary quantization methods, however, the former suffers from significant performance degradation, and the latter incurs computational and storage overhead. We propose UniSVQ, a unified 2-bit quantization framework that bridges scalar and vector quantization by parameterizing codewords as an affine transform of integer lattices. This structure preserves compatibility with optimized integer kernels while retaining much of VQ's flexibility. We further introduce a data-driven block-wise fine-tuning strategy to directly minimize quantization reconstruction error. Extensive experiments across multiple LLM families and zero-shot benchmarks demonstrate that UniSVQ consistently outperforms state-of-the-art SQ methods and achieves performance comparable to advanced VQ methods, while providing higher inference throughput. Codes are publicly available at https://github.com/AI9Stars/UniSVQ.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in KV cache quantization, one of 12 directions in Efficient LLM inference and serving. Builds on 35 of its 38 references that are in the map. First posted on arXiv 9 Jun 2026.

Builds on

35 of 38 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024SpinQuant: LLM quantization with learned rotations26 May

Other references · 34

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  3. 2025Qwen3 Technical Reportcited by 8,820
  4. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  5. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,100
  6. 2016Pointer Sentinel Mixture Modelscited by 3,096
  7. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,883
  8. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,183
  9. 2020MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformerscited by 1,463
  10. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,298
  11. 2023A Simple and Effective Pruning Approach for Large Language Modelscited by 800
  12. 2023LLM-Pruner: On the Structural Pruning of Large Language Modelscited by 650
22 more
  1. 2023OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Modelscited by 424
  2. 2023QuIP: 2-Bit Quantization of Large Language Models With Guaranteescited by 414
  3. 2024QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebookscited by 378
  4. 2023SqueezeLLM: Dense-and-Sparse Quantizationcited by 294
  5. 2024Extreme Compression of Large Language Models via Additive Quantizationcited by 232
  6. 2024RedPajama: an Open Dataset for Training Large Language Modelscited by 182
  7. 2024BiLLM: Pushing the Limit of Post-Training Quantization for LLMscited by 144
  8. 2024FlatQuant: Flatness Matters for LLM Quantizationcited by 131
  9. 2025OstQuant: Refining Large Language Model Quantization with Orthogonal and Scaling Transformations for Better Distribution Fittingcited by 97
  10. 2024Scaling Laws for Precisioncited by 95
  11. 2024QTIP: Quantization with Trellises and Incoherence Processingcited by 91
  12. 2023PB-LLM: Partially Binarized Large Language Modelscited by 83
  13. 2024GPTVQ: The Blessing of Dimensionality for LLM Quantizationcited by 82
  14. 2025SVD-LLM V2: Optimizing Singular Value Truncation for Large Language Model Compressioncited by 67
  15. 2023Optimize Weight Rounding via Signed Gradient Descent for the Quantization of LLMscited by 50
  16. 2025Dobi-SVD: Differentiable SVD for LLM Compression and Some New Perspectivescited by 48
  17. 2025ParetoQ: Improving Scaling Laws in Extremely Low-bit LLM Quantizationcited by 46
  18. 2025Systematic Outliers in Large Language Modelscited by 33
  19. 2025NestQuant: Nested Lattice Quantization for Matrix Products and LLMscited by 23
  20. 2025Low-Precision Training of Large Language Models: Methods, Challenges, and Opportunitiescited by 11
  21. 2025ICQuant: Index Coding enables Low-bit LLM Quantizationcited by 9
  22. 2025CCQ: Convolutional Code for Extreme Low-bit Quantization in LLMscited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026BiSCo-LLM: Lookup-Free Binary Spherical Coding for Extreme Low-Bit Large Language Model Compressioncited by 0
  2. 2026SAGE: Pushing 2-Bit Vector-Quantized LLMs toward Real-World Usabilitycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024VPTQ: Extreme Low-bit Vector Post-Training Quantization for Large Language Models
  2. 2025RaanA: A Fast, Flexible, and Data-Efficient Post-Training Quantization Algorithm
  3. 2024QQQ: Quality Quattuor-Bit Quantization for Large Language Models
  4. 2024I-LLM: Efficient Integer-Only Inference for Fully-Quantized Low-Bit Large Language Models
  5. 2025SplitQuantV2: Enhancing Low-Bit Quantization of LLMs Without GPUs
  6. 2025PCDVQ: Enhancing Vector Quantization for Large Language Models via Polar Coordinate Decoupling
  7. 2025R2Q: Towards Robust 2-Bit Large Language Models via Residual Refinement Quantization
  8. 2025Unifying Block-wise PTQ and Distillation-based QAT for Progressive Quantization toward 2-bit Instruction-Tuned LLMs