GPTQ-intrinsic LoRA: A Near-optimal Algorithm for Low-precision Quantization with Low-rank Adaptation
Not on the map. Why?
Abstract (arXiv)
Post-training quantization is widely used for compressing large neural networks, but aggressive low-bit quantization can significantly degrade model quality. A common remedy is to augment the quantized weights with a low-rank correction, leading to approximations of the form W≈ Q+LR. In this paper, we study this low-precision plus low-rank representation through the layer-wise reconstruction objective |XW-X(Q+LR)|F2, where X is a calibration matrix. We establish, to our knowledge, the first information-theoretic lower bounds for this problem under finite-alphabet and bounded low-rank compensation constraints. We then propose GPTQ-intrinsic LoRA, a training-free algorithm that incorporates the low-rank correction directly into a GPTQ-style quantization pass by appropriately augmenting the calibration Hessian. For the choice L=Vr, where Vr contains the top right singular vectors of X, we prove layer-wise reconstruction error bounds in which the usual GPTQ dependence on |X|F2 is replaced by the rank-r residual |X-Xr|F2, up to regularization terms. Under natural structural assumptions, these bounds match the information-theoretic lower bounds in their dominant scaling, up to constants and mild factors. We also introduce Bid-Up, a fixed-grid quantization refinement step that can be alternated with optimal low-rank compensation with guaranteed non-increasing layer-wise reconstruction error. Experiments on Qwen3 language models and DeiT vision transformers show that GPTQ-intrinsic LoRA improves over GPTQ and GPTQ followed by low-rank compensation, with additional gains from refinement loops.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 61 of its 88 references that are in the map. First posted on arXiv 31 May 2026.
Builds on
61 of 88 references.
References · 61
- 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
- 2025Qwen3 Technical Reportcited by 8,195
- 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
- 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
- 2016Pointer Sentinel Mixture Modelscited by 2,992
- 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,838
- 2019Generating Long Sequences with Sparse Transformerscited by 1,523
- 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,338
- 2023SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shotcited by 1,090
- 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 798
- 2024KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cachecited by 604
38 more
- 2024QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMscited by 598
- 2021Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networkscited by 521
- 2021BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstructioncited by 464
- 2024SpinQuant: LLM quantization with learned rotationscited by 446
- 2022DeiT III: Revenge of the ViTcited by 417
- 2023QuIP: 2-Bit Quantization of Large Language Models With Guaranteescited by 397
- 2021ResNet strikes back: An improved training procedure in timmcited by 365
- 2023SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compressioncited by 359
- 2022Optimal Brain Compression: A Framework for Accurate Post-Training Quantization and Pruningcited by 317
- 2017A Survey of Model Compression and Acceleration for Deep Neural Networkscited by 298
- 2023SqueezeLLM: Dense-and-Sparse Quantizationcited by 285
- 2021Pruning and Quantization for Deep Neural Network Acceleration: A Surveycited by 259
- 2023LoftQ: LoRA-Fine-Tuning-Aware Quantization for Large Language Modelscited by 195
- 2024SVDQuant: Absorbing Outliers by Low-Rank Components for 4-Bit Diffusion Modelscited by 161
- 2023QA-LoRA: Quantization-Aware Low-Rank Adaptation of Large Language Modelscited by 154
- 2021The Low-Rank Simplicity Bias in Deep Networkscited by 150
- 2015Accelerating Very Deep Convolutional Networks for Classification and Detectioncited by 109
- 2023LQ-LoRA: Low-rank Plus Quantized Matrix Decomposition for Efficient Language Model Finetuningcited by 92
- 2023ZeroQuant-V2: Exploring Post-training Quantization in LLMs from Comprehensive Study to Low Rank Compensationcited by 78
- 2020An Overview of Neural Network Compressioncited by 59
- 2024Compressing Large Language Models using Low Rank and Low Precision Decompositioncited by 58
- 2023LoRAPrune: Structured Pruning Meets Low-Rank Parameter-Efficient Fine-Tuningcited by 53
- 2023Pruning vs Quantization: Which is Better?cited by 45
- 2024LQER: Low-Rank Quantization Error Reconstruction for LLMscited by 39
- 2022Post-training Quantization for Neural Networks with Provable Guaranteescited by 32
- 2022Randomly pivoted Cholesky: Practical approximation of a kernel matrix with few entry evaluationscited by 31
- 2025Bridging the Gap Between Promise and Performance for Microscaling FP4 Quantizationcited by 30
- 2025Four Over Six: More Accurate NVFP4 Quantization with Adaptive Block Scalingcited by 29
- 2024ApiQ: Finetuning of 2-Bit Quantized Large Language Modelcited by 27
- 2024MagR: Weight Magnitude Reduction for Enhancing Post-Training Quantizationcited by 26
- 2025Qronos: Correcting the Past by Shaping the Future... in Post-Training Quantizationcited by 24
- 2020A Greedy Algorithm for Quantizing Neural Networkscited by 21
- 2024QERA: an Analytical Framework for Quantization Error Reconstructioncited by 21
- 2024OATS: Outlier-Aware Pruning Through Sparse and Low Rank Decompositioncited by 19
- 2024LLMCBench: Benchmarking Large Language Model Compression for Efficient Deploymentcited by 16
- 2025INT v.s. FP: A Comprehensive Study of Fine-Grained Low-bit Quantization Formatscited by 16
- 2025QSVD: Efficient Low-rank Approximation for Unified Query-Key-Value Weight Compression in Low-Precision Vision-Language Modelscited by 15
- 2021Sparse Plus Low Rank Matrix Decomposition: A Discrete Optimization Approachcited by 14
50 most cited of 61 All 61 (JSON)
Built on by
Citing papers per year: 2026: 1.
1 paper.