mlsanityData to 7 Oct 2026

LQER: Low-Rank Quantization Error Reconstruction for LLMs

arXiv 2402.02446 ↗First posted 4 Feb 2024cs.LG cs.CLcited by 39

Not on the map. Why?

Abstract (arXiv)

Post-training quantization of Large Language Models (LLMs) is challenging. In this work, we introduce Low-rank Quantization Error Reduction (LQER), which combines quantization and low-rank approximation to recover the model capability. LQER leverages an activation-induced scale matrix to drive the singular value distribution of quantization error towards a desirable distribution, which enables nearly-lossless W4A8 quantization on various LLMs and downstream tasks without the need for knowledge distillation, grid search, or gradient-base iterative optimization. Unlike existing methods, the computation pattern of LQER eliminates the need for specialized Scatter and Gather processes to collect high-precision weights from irregular memory locations. Our W4A8 LLMs achieve near-lossless performance on six popular downstream tasks, while using 1.36× fewer hardware resources than the leading state-of-the-art method. We open-source our framework at https://github.com/ChengZhang-98/lqer

Key facts. Cited by 39 papers in the map (data through 7 Oct 2026). Builds on 39 of its 59 references that are in the map. First posted on arXiv 4 Feb 2024.

Builds on

39 of 59 references.

References · 39

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  5. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  6. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  7. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  8. 2020Longformer: The Long-Document Transformercited by 3,362
  9. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  10. 2023Mistral 7Bcited by 3,065
  11. 2016Pointer Sentinel Mixture Modelscited by 2,992
  12. 2022Training Compute-Optimal Large Language Modelscited by 2,927
27 more
  1. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,818
  2. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,140
  3. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,114
  4. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,968
  5. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,838
  6. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,338
  7. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,234
  8. 2016The LAMBADA dataset: Word prediction requiring a broad discourse contextcited by 960
  9. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 881
  10. 2021A White Paper on Neural Network Quantizationcited by 453
  11. 2023OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Modelscited by 403
  12. 2023QuIP: 2-Bit Quantization of Large Language Models With Guaranteescited by 397
  13. 2023SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compressioncited by 359
  14. 2023LLM-QAT: Data-Free Quantization Aware Training for Large Language Modelscited by 320
  15. 2022Estimating the Carbon Footprint of BLOOM, a 176B Parameter Language Modelcited by 264
  16. 2023LoftQ: LoRA-Fine-Tuning-Aware Quantization for Large Language Modelscited by 195
  17. 2022Outlier Suppression: Pushing the Limit of Low-bit Transformer Language Modelscited by 176
  18. 2023Quantizable Transformers: Removing Outliers by Helping Attention Heads Do Nothingcited by 147
  19. 2023Microscaling Data Formats for Deep Learningcited by 132
  20. 2023QLLM: Accurate and Efficient Low-Bitwidth Quantization for Large Language Modelscited by 63
  21. 2023With Shared Microexponents, A Little Shifting Goes a Long Waycited by 47
  22. 2023FlexRound: Learnable Rounding based on Element-wise Division for Post-Training Quantizationcited by 46
  23. 2018Training DNNs with Hybrid Block Floating Pointcited by 39
  24. 2021FAST: DNN Training Under Variable Precision Block Floating Point with Stochastic Roundingcited by 31
  25. 2023Enhancing Computation Efficiency in Large Language Models through Weight and Activation Quantizationcited by 20
  26. 2023Revisiting Block-based Quantisation: What is Important for Sub-8-bit LLM Inference?cited by 8
  27. 2022Long-Range Zero-Shot Generative Deep Network Quantizationcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 13; 2025: 8; 2026: 18.

39 papers.

Sort:

By area:

  1. 2024Low-Rank Quantization-Aware Training for LLMscited by 42
  2. 2024Low-Rank Adaptation for Foundation Models: A Comprehensive Reviewcited by 35
  3. 2024A Survey of Low-bit Large Language Models: Basics, Systems, and Algorithmscited by 30
  4. 2024QERA: an Analytical Framework for Quantization Error Reconstructioncited by 21
  5. 2024Low-Bit Quantization Favors Undertrained LLMs: Scaling Laws for Quantized LLMs with 100T Training Tokenscited by 18
  6. 2024SLiM: One-shot Quantization and Sparsity with Low-rank Approximation for LLM Weight Compressioncited by 12
  7. 2025Low-bit Model Quantization for Deep Neural Networks: A Surveycited by 11
  8. 2025LittleBit: Ultra Low-Bit Quantization via Latent Factorizationcited by 11
  9. 2026LLaVA-FA: Learning Fourier Approximation for Compressing Large Multimodal Modelscited by 10
  10. 2024RILQ: Rank-Insensitive LoRA-based Quantization Error Compensation for Boosting 2-bit Large Language Model Accuracycited by 9
  11. 2024ASER: Activation Smoothing and Error Reconstruction for Large Language Model Quantizationcited by 8
  12. 2025A3 : an Analytical Low-Rank Approximation Framework for Attentioncited by 7
27 more
  1. 2024EoRA: Fine-tuning-free Compensation for Compressed LLM with Eigenspace Low-Rank Approximationcited by 6
  2. 2024Towards Low-bit Communication for Tensor Parallel LLM Inferencecited by 6
  3. 2024CDQuant: Greedy Coordinate Descent for Accurate LLM Quantizationcited by 4
  4. 2025Bandwidth-Efficient Adaptive Mixture-of-Experts via Low-Rank Compensationcited by 3
  5. 2026LoPRo: Enhancing Low-Rank Quantization via Permuted Block-Wise Rotationcited by 3
  6. 2026Quant Experts: Token-aware Adaptive Error Reconstruction with Mixture of Experts for Large Vision-Language Models Quantizationcited by 3
  7. 2024Low-Rank Correction for Quantized LLMscited by 2
  8. 2026FLRQ: Faster LLM Quantization with Flexible Low-Rank Matrix Sketchingcited by 2
  9. 2026EdgeRazor: A Lightweight Framework for Large Language Models via Mixed-Precision Quantization-Aware Distillationcited by 2
  10. 2024Scaling Laws For Mixed Quantizationcited by 1
  11. 2025Efficient Large Language Model Inference with Neural Block Linearizationcited by 1
  12. 2025Assigning Distinct Roles to Quantized and Low-Rank Matrices Toward Optimal Weight Decompositioncited by 1
  13. 2025On Information Geometry and Iterative Optimization in Model Compression: Operator Factorizationcited by 1
  14. 2025A Comprehensive Evaluation on Quantization Techniques for Large Language Modelscited by 1
  15. 2026Preserve-Then-Quantize: Balancing Rank Budgets for Quantization Error Reconstruction in LLMscited by 1
  16. 2026SERQ: Saliency-Aware Low-Rank Error Reconstruction for LLM Quantizationcited by 1
  17. 2026D-QRELO: Training- and Data-Free Delta Compression for Large Language Models via Quantization and Residual Low-Rank Approximationcited by 1
  18. 2026GPTQ-intrinsic LoRA: A Near-optimal Algorithm for Low-precision Quantization with Low-rank Adaptationcited by 1
  19. 2026MixQuant: Adaptive Mixed-Precision Quantization for Large Language Modelscited by 1
  20. 2026GlowQ: Group-Shared LOw-Rank Approximation for Quantized LLMscited by 0
  21. 2026TileQ: Efficient Low-Rank Quantization of Mixture-of-Experts with 2D Tilingcited by 0
  22. 2026SPEAR: A System for Post-Quantization Error-Adaptive Recovery Enabling Efficient Low-Bit LLM Servingcited by 0
  23. 2026Parameter Golf: What Really Works?cited by 0
  24. 2026Language-Conditional Dequantization: Recovering What Quantization Steals from Non-English Languagescited by 0
  25. 2026Activation-Weighted Seeded Residual Coding for Low-Bit LLM Weight Repaircited by 0
  26. 2026A Multi-Engine Dataflow for MoE Decoding on Scratchpad-Based Tensor Acceleratorscited by 0
  27. 2026Quantization Error Is Spectrally Flat: A Single Random Probe Is a Calibrated, Data-Free Sensitivity Estimator, with Application to Budget-Targeted Mixed-Precision Quantizationcited by 0