mlsanityData to 7 Oct 2026

Accurate LoRA-Finetuning Quantization of LLMs via Information Retention

arXiv 2402.05445 ↗First posted 8 Feb 2024cs.LG cs.CLcited by 40

Not on the map. Why?

Abstract (arXiv)

The LoRA-finetuning quantization of LLMs has been extensively studied to obtain accurate yet compact LLMs for deployment on resource-constrained hardware. However, existing methods cause the quantized LLM to severely degrade and even fail to benefit from the finetuning of LoRA. This paper proposes a novel IR-QLoRA for pushing quantized LLMs with LoRA to be highly accurate through information retention. The proposed IR-QLoRA mainly relies on two technologies derived from the perspective of unified information: (1) statistics-based Information Calibration Quantization allows the quantized parameters of LLM to retain original information accurately; (2) finetuning-based Information Elastic Connection makes LoRA utilizes elastic representation transformation with diverse information. Comprehensive experiments show that IR-QLoRA can significantly improve accuracy across LLaMA and LLaMA2 families under 2-4 bit-widths, e.g., 4- bit LLaMA-7B achieves 1.4% improvement on MMLU compared with the state-of-the-art methods. The significant performance gain requires only a tiny 0.31% additional time consumption, revealing the satisfactory efficiency of our IR-QLoRA. We highlight that IR-QLoRA enjoys excellent versatility, compatible with various frameworks (e.g., NormalFloat and Integer quantization) and brings general accuracy gains. The code is available at https://github.com/htqin/ir-qlora.

Key facts. Cited by 40 papers in the map (data through 7 Oct 2026). Builds on 35 of its 45 references that are in the map. First posted on arXiv 8 Feb 2024.

Builds on

35 of 45 references.

References · 35

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  5. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  6. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  7. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  8. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  9. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,818
  10. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  11. 2023A Survey of Large Language Modelscited by 2,537
  12. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,140
23 more
  1. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,114
  2. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,838
  3. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,338
  4. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,234
  5. 2023A Survey on Evaluation of Large Language Modelscited by 1,227
  6. 2023SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shotcited by 1,090
  7. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 881
  8. 2023A Simple and Effective Pruning Approach for Large Language Modelscited by 776
  9. 2023The Flan Collection: Designing Data and Methods for Effective Instruction Tuningcited by 734
  10. 2022Towards Reasoning in Large Language Models: A Surveycited by 593
  11. 2023OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Modelscited by 403
  12. 20218-bit Optimizers via Block-wise Quantizationcited by 395
  13. 2023LLM-QAT: Data-Free Quantization Aware Training for Large Language Modelscited by 320
  14. 2023A Survey on Model Compression for Large Language Modelscited by 263
  15. 2023LoftQ: LoRA-Fine-Tuning-Aware Quantization for Large Language Modelscited by 195
  16. 2023MiniLLM: On-Policy Distillation of Large Language Modelscited by 129
  17. 2022Explanations from Large Language Models Make Small Reasoners Bettercited by 123
  18. 2023Memory-Efficient Fine-Tuning of Compressed Large Language Models via sub-4-bit Integer Quantizationcited by 97
  19. 2020Compressing Large-Scale Transformer-Based Models: A Case Study on BERTcited by 95
  20. 2023Do Emergent Abilities Exist in Quantized Large Language Models: An Empirical Studycited by 43
  21. 2022In-context Learning Distillation: Transferring Few-shot Learning Ability of Pre-trained Language Modelscited by 36
  22. 2023A Survey of Techniques for Optimizing Transformer Inferencecited by 35
  23. 2021Distribution-sensitive Information Retention for Accurate Binary Neural Networkcited by 25

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 14; 2025: 19; 2026: 7.

40 papers.

Sort:

By area:

  1. 2024EfficientQAT: Efficient Quantization-Aware Training for Large Language Modelscited by 123
  2. 2024Catastrophic Failure of LLM Unlearning via Quantizationcited by 76
  3. 2024An empirical study of LLaMA3 quantization: from LLMs to MLLMscited by 63
  4. 2024SliM-LLM: Salience-Driven Mixed-Precision Quantization for Large Language Modelscited by 61
  5. 2025Sustainable LLM Inference for Edge AI: Evaluating Quantized LLMs for Energy Efficiency, Output Accuracy, and Inference Latencycited by 32
  6. 2024A Survey of Low-bit Large Language Models: Basics, Systems, and Algorithmscited by 30
  7. 2024SpikeLLM: Scaling up Spiking Neural Network to Large Language Models via Saliency-based Spikingcited by 29
  8. 2025Quantization Meets dLLMs: A Systematic Study of Post-training Quantization for Diffusion LLMscited by 21
  9. 2026Flow caching for autoregressive video generationcited by 19
  10. 2025Investigating the Impact of Quantization Methods on the Safety and Reliability of Large Language Modelscited by 17
  11. 2024RoLoRA: Fine-tuning Rotated Outlier-free LLMs for Effective Weight-Activation Quantizationcited by 13
  12. 2025ORANSight-2.0: Foundational LLMs for O-RANcited by 13
28 more
  1. 2024BinaryDM: Accurate Weight Binarization for Efficient Diffusion Modelscited by 11
  2. 2024BiDM: Pushing the Limit of Quantization for Diffusion Modelscited by 11
  3. 2024Theoretical Insights into Fine-Tuning Attention Mechanism: Generalization and Optimizationcited by 9
  4. 2024IntLoRA: Integral Low-rank Adaptation of Quantized Diffusion Modelscited by 9
  5. 2026What Makes Low-Bit Quantization-Aware Training Work for Reasoning LLMs? A Systematic Studycited by 9
  6. 2025LowRA: Accurate and Efficient LoRA Fine-Tuning of LLMs under 2 Bitscited by 6
  7. 2025Towards Accurate Binary Spiking Neural Networks: Learning with Adaptive Gradient Modulation Mechanismcited by 5
  8. 2025Event-Priori-Based Vision-Language Model for Efficient Visual Understandingcited by 5
  9. 2024Sketch to Adapt: Fine-Tunable Sketches for Efficient LLM Adaptationcited by 3
  10. 2025Enhancing Ultra-Low-Bit Quantization of Large Language Models Through Saliency-Aware Partial Retrainingcited by 3
  11. 2024On-Device LLMs for SMEs: Challenges and Opportunitiescited by 2
  12. 2025End-to-End On-Device Quantization-Aware Training for LLMs at Inference Costcited by 2
  13. 2025Towards Fast LLM Fine-tuning through Zeroth-Order Optimization with Projected Gradient-Aligned Perturbationscited by 2
  14. 2026StableQAT: Stable Quantization-Aware Training at Ultra-Low Bitwidthscited by 2
  15. 2026Low-Rank Adaptation Redux for Large Modelscited by 2
  16. 2025TeTRA-VPR: A Ternary Transformer Approach for Compact Visual Place Recognitioncited by 1
  17. 2025Spiritual-LLM : Gita Inspired Mental Health Therapy In the Era of LLMscited by 1
  18. 2025BiVM: Accurate Binarized Neural Network for Efficient Video Mattingcited by 1
  19. 2025APT-LLM: Exploiting Arbitrary-Precision Tensor Core Computing for LLM Accelerationcited by 1
  20. 2025On-the-Fly Adaptation to Quantization: Configuration-Aware LoRA for Efficient Fine-Tuning of Quantized LLMscited by 1
  21. 2025FALQON: Accelerating LoRA Fine-tuning with Low-Bit Floating-Point Arithmeticcited by 1
  22. 2025Towards Efficient VLMs: Information-Theoretic Driven Compression via Adaptive Structural Pruningcited by 1
  23. 2024Accurate and Efficient Fine-Tuning of Quantized Large Language Models Through Optimal Balancecited by 0
  24. 2025Reducing Storage of Pretrained Neural Networks by Rate-Constrained Quantization and Entropy Codingcited by 0
  25. 2025Reservoir Computing inspired Matrix Multiplication-free Language Modelcited by 0
  26. 2026AQLoRA: A Zero-Search Recipe for Fast Quantized LoRA Fine-Tuningcited by 0
  27. 2026Scale-QLoRA: Code-Invariant Adapter Merging for Native 4-bit Microscaling LLMscited by 0
  28. 2026Automated Regulatory Compliance Question Answering in Financial Services with Domain-Adapted Retrieval-Augmented Generationcited by 0