mlsanityData to 14 Oct 2026

RILQ: Rank-Insensitive LoRA-based Quantization Error Compensation for Boosting 2-bit Large Language Model Accuracy

arXiv 2412.01129 ↗First posted 2 Dec 2024cs.LG cs.AIcited by 9

Not on the map. Why?

Abstract (arXiv)

Low-rank adaptation (LoRA) has become the dominant method for parameter-efficient LLM fine-tuning, with LoRA-based quantization error compensation (LQEC) emerging as a powerful tool for recovering accuracy in compressed LLMs. However, LQEC has underperformed in sub-4-bit scenarios, with no prior investigation into understanding this limitation. We propose RILQ (Rank-Insensitive LoRA-based Quantization Error Compensation) to understand fundamental limitation and boost 2-bit LLM accuracy. Based on rank analysis revealing model-wise activation discrepancy loss's rank-insensitive nature, RILQ employs this loss to adjust adapters cooperatively across layers, enabling robust error compensation with low-rank adapters. Evaluations on LLaMA-2 and LLaMA-3 demonstrate RILQ's consistent improvements in 2-bit quantized inference across various state-of-the-art quantizers and enhanced accuracy in task-specific fine-tuning. RILQ maintains computational efficiency comparable to existing LoRA methods, enabling adapter-merged weight-quantized LLM inference with significantly enhanced accuracy, making it a promising approach for boosting 2-bit LLM performance. Our code is available at https://github.com/aiha-lab/RILQ.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Builds on 31 of its 49 references that are in the map. First posted on arXiv 2 Dec 2024.

Builds on

31 of 49 references.

References · 31

  1. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  2. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  3. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  4. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,100
  5. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  6. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  7. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,883
  8. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,724
  9. 2023Code Llama: Open Foundation Models for Codecited by 2,139
  10. 2024Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Surveycited by 656
  11. 2024QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMscited by 634
  12. 2022ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformerscited by 526
19 more
  1. 2021BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstructioncited by 479
  2. 2023Benchmarking Large Language Models for News Summarizationcited by 445
  3. 2023OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Modelscited by 424
  4. 2024QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebookscited by 378
  5. 2023LoraHub: Efficient Cross-Task Generalization via Dynamic LoRA Compositioncited by 364
  6. 2024Extreme Compression of Large Language Models via Additive Quantizationcited by 232
  7. 2022Outlier Suppression: Pushing the Limit of Low-bit Transformer Language Modelscited by 181
  8. 2023S-LoRA: Serving Thousands of Concurrent LoRA Adapterscited by 152
  9. 2023OliVe: Accelerating Large Language Models via Hardware-friendly Outlier-Victim Pair Quantizationcited by 92
  10. 2023LQ-LoRA: Low-rank Plus Quantized Matrix Decomposition for Efficient Language Model Finetuningcited by 92
  11. 2023ZeroQuant-V2: Exploring Post-training Quantization in LLMs from Comprehensive Study to Low Rank Compensationcited by 79
  12. 2023Parameter-Efficient Fine-Tuning Design Spacescited by 67
  13. 2023Outlier Suppression+: Accurate quantization of large language models by equivalent and optimal shifting and scalingcited by 61
  14. 2023FlexRound: Learnable Rounding based on Element-wise Division for Post-Training Quantizationcited by 48
  15. 2024LQER: Low-Rank Quantization Error Reconstruction for LLMscited by 41
  16. 2023LoRAShear: Efficient Large Language Model Structured Pruning and Knowledge Recoverycited by 38
  17. 2023INT2.1: Towards Fine-Tunable Quantized Large Language Models with Error Correction through Low-Rank Adaptationcited by 20
  18. 2023Token-Scaled Logit Distillation for Ternary Weight Generative Language Modelscited by 20
  19. 2024RoLoRA: Fine-tuning Rotated Outlier-free LLMs for Effective Weight-Activation Quantizationcited by 13

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 7.

9 papers.

  1. 2025Balancing Fidelity and Plasticity: Aligning Mixed-Precision Fine-Tuning with Linguistic Hierarchiescited by 5
  2. 2026LoPRo: Enhancing Low-Rank Quantization via Permuted Block-Wise Rotationcited by 3
  3. 2026FLRQ: Faster LLM Quantization with Flexible Low-Rank Matrix Sketchingcited by 2
  4. 2026LFQ: Logit-aware Final-block Quantization for Boosting the Generation Quality of Low-Bit Quantized LLMscited by 2
  5. 2025FALQON: Accelerating LoRA Fine-tuning with Low-Bit Floating-Point Arithmeticcited by 1
  6. 2026Preserve-Then-Quantize: Balancing Rank Budgets for Quantization Error Reconstruction in LLMscited by 1
  7. 2026LAQuant: A Simple Overhead-free Large Reasoning Model Quantization by Layer-wise Lookahead Losscited by 1
  8. 2026BiSCo-LLM: Lookup-Free Binary Spherical Coding for Extreme Low-Bit Large Language Model Compressioncited by 0
  9. 2026Language-Conditional Dequantization: Recovering What Quantization Steals from Non-English Languagescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026SERQ: Saliency-Aware Low-Rank Error Reconstruction for LLM Quantization
  2. 2026Recover-LoRA for Aggressive Quantization: Reclaiming Accuracy in 2-Bit Language Models via Low-Rank Adaptation with Knowledge Distillation on Synthetic Data
  3. 2024L4Q: Parameter Efficient Quantization-Aware Fine-Tuning on Large Language Models
  4. 2025SDQ-LLM: Sigma-Delta Quantization for 1-bit LLMs of any size
  5. 2026Align, Then Correct: Training-Free Two-Stage Low-Rank Compensation for Extremely Quantized Large Language Models
  6. 2025CLoQ: Enhancing Fine-Tuning of Quantized LLMs via Calibrated LoRA Initialization
  7. 2023QA-LoRA: Quantization-Aware Low-Rank Adaptation of Large Language Models
  8. 2024Low-Rank Quantization-Aware Training for LLMs