mlsanityData to 7 Oct 2026

EfficientQAT: Efficient Quantization-Aware Training for Large Language Models

arXiv 2407.11062 ↗First posted 10 Jul 2024cs.LG cs.AI cs.CLcited by 123

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) are crucial in modern natural language processing and artificial intelligence. However, they face challenges in managing their significant memory requirements. Although quantization-aware training (QAT) offers a solution by reducing memory consumption through low-bit representations with minimal accuracy loss, it is impractical due to substantial training resources. To address this, we propose Efficient Quantization-Aware Training (EfficientQAT), a more feasible QAT algorithm. EfficientQAT involves two consecutive phases: Block-wise training of all parameters (Block-AP) and end-to-end training of quantization parameters (E2E-QP). To the best of our knowledge, Block-AP is the first method to enable direct training of all parameters in a block-wise manner, reducing accuracy loss in low-bit scenarios by enhancing the solution space during optimization. E2E-QP then trains only the quantization parameters (step sizes) end-to-end, further improving the performance of quantized models by considering interactions among all sub-modules. Extensive experiments demonstrate that EfficientQAT outperforms previous quantization methods across a range of models, including base LLMs, instruction-tuned LLMs, and multimodal LLMs, with scales from 7B to 70B parameters at various quantization bits. For instance, EfficientQAT obtains a 2-bit Llama-2-70B model on a single A100-80GB GPU in 41 hours, with less than 3 points accuracy degradation compared to the full precision (69.48 vs. 72.41). Code is available at https://github.com/OpenGVLab/EfficientQAT.

Key facts. Cited by 123 papers in the map (data through 7 Oct 2026). Builds on 65 of its 79 references that are in the map. First posted on arXiv 10 Jul 2024.

Builds on

65 of 79 references.

References · 65

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2023Visual Instruction Tuningcited by 7,792
  5. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  6. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  7. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  8. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  9. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  10. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,818
  11. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  12. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,140
38 more
  1. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,105
  2. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,031
  3. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,881
  4. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
  5. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,838
  6. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,624
  7. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,338
  8. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,234
  9. 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,221
  10. 2019Learned Step Size Quantizationcited by 604
  11. 2020Up or Down? Adaptive Rounding for Post-Training Quantizationcited by 600
  12. 2024QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMscited by 598
  13. 2021BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstructioncited by 464
  14. 2023OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Modelscited by 403
  15. 2023QuIP: 2-Bit Quantization of Large Language Models With Guaranteescited by 397
  16. 2023SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compressioncited by 359
  17. 2024QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebookscited by 354
  18. 2023LLM-QAT: Data-Free Quantization Aware Training for Large Language Modelscited by 320
  19. 2023Tool Learning with Foundation Modelscited by 302
  20. 2024The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bitscited by 289
  21. 2023SqueezeLLM: Dense-and-Sparse Quantizationcited by 285
  22. 2023Atom: Low-bit Quantization for Efficient and Accurate LLM Servingcited by 255
  23. 2024Extreme Compression of Large Language Models via Additive Quantizationcited by 223
  24. 2023LVLM-eHub: A Comprehensive Evaluation Benchmark for Large Vision-Language Modelscited by 217
  25. 2024QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Servingcited by 200
  26. 2023LoftQ: LoRA-Fine-Tuning-Aware Quantization for Large Language Modelscited by 195
  27. 2024MMT-Bench: A Comprehensive Multimodal Benchmark for Evaluating Large Vision-Language Models Towards Multitask AGIcited by 192
  28. 2020LSQ+: Improving low-bit quantization through learnable offsets and better initializationcited by 180
  29. 2022Outlier Suppression: Pushing the Limit of Low-bit Transformer Language Modelscited by 176
  30. 2023QA-LoRA: Quantization-Aware Low-Rank Adaptation of Large Language Modelscited by 154
  31. 2024LLM Inference Unveiled: Survey and Roofline Model Insightscited by 147
  32. 2023RPTQ: Reorder-based Post-training Quantization for Large Language Modelscited by 112
  33. 2023Memory-Efficient Fine-Tuning of Compressed Large Language Models via sub-4-bit Integer Quantizationcited by 97
  34. 2023LQ-LoRA: Low-rank Plus Quantized Matrix Decomposition for Efficient Language Model Finetuningcited by 92
  35. 2023PB-LLM: Partially Binarized Large Language Modelscited by 80
  36. 2024BitDistiller: Unleashing the Potential of Sub-4-Bit LLMs via Self-Distillationcited by 80
  37. 2024OneBit: Towards Extremely Low-bit Large Language Modelscited by 77
  38. 2023QLLM: Accurate and Efficient Low-Bitwidth Quantization for Large Language Modelscited by 63

50 most cited of 65 All 65 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 16; 2025: 47; 2026: 60.

123 papers.

Sort:

By area:

  1. 2025ParetoQ: Improving Scaling Laws in Extremely Low-bit LLM Quantizationcited by 44
  2. 2024Mixture Compressor for Mixture-of-Experts LLMs Gains Morecited by 38
  3. 2024A Comprehensive Study on Quantization Techniques for Large Language Modelscited by 34
  4. 20241.58-bit FLUXcited by 31
  5. 2024ARB-LLM: Alternating Refined Binarizations for Large Language Modelscited by 30
  6. 2025Parallel Loop Transformer for Efficient Test-Time Computation Scalingcited by 23
  7. 2025Quantization Meets dLLMs: A Systematic Study of Post-training Quantization for Diffusion LLMscited by 21
  8. 2025A Survey on Efficient Vision-Language Modelscited by 19
  9. 2026Flow caching for autoregressive video generationcited by 19
  10. 2026AutoSOTA: An End-to-End Automated Research System for State-of-the-Art AI Model Discoverycited by 17
  11. 2025FPTQuant: Function-Preserving Transforms for LLM Quantizationcited by 16
  12. 2025INT v.s. FP: A Comprehensive Study of Fine-Grained Low-bit Quantization Formatscited by 16
38 more
  1. 2025ParoQuant: Pairwise Rotation Quantization for Efficient Reasoning LLM Inferencecited by 16
  2. 2025Scaling Law for Quantization-Aware Trainingcited by 15
  3. 2025QeRL: Beyond Efficiency -- Quantization-enhanced Reinforcement Learning for LLMscited by 13
  4. 2024BitMoD: Bit-serial Mixture-of-Datatype LLM Accelerationcited by 12
  5. 2024PrefixQuant: Eliminating Outliers by Prefixed Tokens for Large Language Models Quantizationcited by 11
  6. 2025Matryoshka Quantizationcited by 11
  7. 2025Low-Precision Training of Large Language Models: Methods, Challenges, and Opportunitiescited by 11
  8. 2025MoE-CAP: Benchmarking Cost, Accuracy and Performance of Sparse Mixture-of-Experts Systemscited by 11
  9. 2026What Makes Low-Bit Quantization-Aware Training Work for Reasoning LLMs? A Systematic Studycited by 9
  10. 2024Anda: Unlocking Efficient LLM Inference with a Variable-Length Grouped Activation Data Formatcited by 8
  11. 2025ButterflyQuant: Ultra-low-bit LLM Quantization through Learnable Orthogonal Butterfly Transformscited by 8
  12. 2024CRVQ: Channel-Relaxed Vector Quantization for Extreme Compression of LLMscited by 7
  13. 2025Unifying Block-wise PTQ and Distillation-based QAT for Progressive Quantization toward 2-bit Instruction-Tuned LLMscited by 7
  14. 2025GPLQ: A General, Practical, and Lightning QAT Method for Vision Transformerscited by 7
  15. 2025Layer-wise dynamic rank for compressing large language modelscited by 6
  16. 2026Embodied Foundation Models at the Edge: A Survey of Deployment Constraints and Mitigation Strategiescited by 6
  17. 2025On Accelerating Edge AI: Optimizing Resource-Constrained Environmentscited by 5
  18. 2025iFairy: the First 2-bit Complex LLM with All Parameters in ±1, ± icited by 5
  19. 2026MASQuant: Modality-Aware Smoothing Quantization for Multimodal Large Language Modelscited by 5
  20. 2026GEMQ: Global Expert-Level Mixed-Precision Quantization for MoE LLMscited by 5
  21. 2026CAT-Q: Cost-efficient and Accurate Ternary Quantization for LLMscited by 5
  22. 2024Taming Sensitive Weights : Noise Perturbation Fine-tuning for Robust LLM Quantizationcited by 4
  23. 2025QUAD: Quantization and Parameter-Efficient Tuning of LLM with Activation Decompositioncited by 4
  24. 2025Compute-Optimal Quantization-Aware Trainingcited by 4
  25. 2025Mixed-Precision Quantization for Language Models: Techniques and Prospectscited by 4
  26. 2026DynamiQ: Accelerating Gradient Synchronization using Compressed Multi-hop All-reducecited by 4
  27. 2026GSQ: Highly-Accurate Low-Precision Scalar Quantization for LLMs via Gumbel-Softmax Samplingcited by 4
  28. 2024From Noise to Nuance: Advances in Deep Generative Image Modelscited by 3
  29. 2025A Survey on Memory-Efficient Transformer-Based Model Training in AI for Sciencecited by 3
  30. 2025Enhancing Ultra-Low-Bit Quantization of Large Language Models Through Saliency-Aware Partial Retrainingcited by 3
  31. 2025LoTA-QAF: Lossless Ternary Adaptation for Quantization-Aware Fine-Tuningcited by 3
  32. 2025Exploring Layer-wise Information Effectiveness for Post-Training Quantization in Small Language Modelscited by 3
  33. 2026ECO: Quantized Training without Full-Precision Master Weightscited by 3
  34. 2026FreeAct: Freeing Activations for LLM Quantizationcited by 3
  35. 2026ReMoE: Boosting Expert Reuse through Router Fine-Tuning in Memory-Constrained MoE LLM Inferencecited by 3
  36. 2024Art and Science of Quantizing Large-Scale Models: A Comprehensive Overviewcited by 2
  37. 2024An Efficient Matrix Multiplication Algorithm for Accelerating Inference in Binary and Ternary Neural Networkscited by 2
  38. 2025FedHQ: Hybrid Runtime Quantization for Federated Learningcited by 2

50 most cited of 123 All 123 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026QuBLAST: A Framework for Quantizing Large Language Models with Block-Level Compression Approach and Activation Scaling Strategy
  2. 2025RoSTE: An Efficient Quantization-Aware Supervised Fine-Tuning Approach for Large Language Models
  3. 2025R2Q: Towards Robust 2-Bit Large Language Models via Residual Refinement Quantization
  4. 2024WKVQuant: Quantizing Weight and Key/Value Cache for Large Language Models Gains More
  5. 2025Tequila: Trapping-free Ternary Quantization for Large Language Models
  6. 2024Evaluating Quantized Large Language Models
  7. 2025BlockDialect: Block-wise Fine-grained Mixed Format Quantization for Energy-Efficient LLM Inference
  8. 2025PTQTP: Post-Training Quantization to Trit-Planes for Large Language Models