LCQ: Low-Rank Codebook based Quantization for Large Language Models
Not on the map. Why?
Abstract (arXiv)
Large language models~(LLMs) have recently demonstrated promising performance in many tasks. However, the high storage and computational cost of LLMs has become a challenge for deploying LLMs. Weight quantization has been widely used for model compression, which can reduce both storage and computational cost. Most existing weight quantization methods for LLMs use a rank-one codebook for quantization, which results in substantial accuracy loss when the compression ratio is high. In this paper, we propose a novel weight quantization method, called low-rank codebook based quantization~(LCQ), for LLMs. LCQ adopts a low-rank codebook, the rank of which can be larger than one, for quantization. Experiments show that LCQ can achieve better accuracy than existing methods with a negligibly extra storage cost.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 31 of its 39 references that are in the map. First posted on arXiv 31 May 2024.
Builds on
31 of 39 references.
References · 31
- 2017Attention Is All You Needcited by 59,932
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
- 2023Visual Instruction Tuningcited by 7,792
- 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
- 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
- 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
- 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
- 2016Pointer Sentinel Mixture Modelscited by 2,992
- 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,818
19 more
- 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,346
- 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
- 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,838
- 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,338
- 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 881
- 2022GLM-130B: An Open Bilingual Pre-trained Modelcited by 837
- 2022ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformerscited by 517
- 2021BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstructioncited by 464
- 2023OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Modelscited by 403
- 2023SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compressioncited by 359
- 2022Optimal Brain Compression: A Framework for Accurate Post-Training Quantization and Pruningcited by 317
- 2023SqueezeLLM: Dense-and-Sparse Quantizationcited by 285
- 2023LoftQ: LoRA-Fine-Tuning-Aware Quantization for Large Language Modelscited by 195
- 2022QDrop: Randomly Dropping Quantization for Extremely Low-bit Post-Training Quantizationcited by 175
- 2023QA-LoRA: Quantization-Aware Low-Rank Adaptation of Large Language Modelscited by 154
- 2023RPTQ: Reorder-based Post-training Quantization for Large Language Modelscited by 112
- 2023Memory-Efficient Fine-Tuning of Compressed Large Language Models via sub-4-bit Integer Quantizationcited by 97
- 2021Nonuniform-to-Uniform Quantization: Towards Accurate Quantization via Generalized Straight-Through Estimationcited by 78
- 2021Towards Efficient Post-training Quantization of Pre-trained Language Modelscited by 37
Built on by
Citing papers per year: 2025: 1.
1 paper.
- 2025Neural Weight Compression for Language Modelscited by 0
- 2025Neural Weight Compression for Language Models13 Oct 2025