mlsanityData to 7 Oct 2026

GuidedQuant: Large Language Model Quantization via Exploiting End Loss Guidance

arXiv 2505.07004 ↗First posted 11 May 2025cs.LGcited by 28

Not on the map. Why?

Abstract (arXiv)

Post-training quantization is a key technique for reducing the memory and inference latency of large language models by quantizing weights and activations without requiring retraining. However, existing methods either (1) fail to account for the varying importance of hidden features to the end loss or, when incorporating end loss, (2) neglect the critical interactions between model weights. To address these limitations, we propose GuidedQuant, a novel quantization approach that integrates gradient information from the end loss into the quantization objective while preserving cross-weight dependencies within output channels. GuidedQuant consistently boosts the performance of state-of-the-art quantization methods across weight-only scalar, weight-only vector, and weight-and-activation quantization. Additionally, we introduce a novel non-uniform scalar quantization algorithm, which is guaranteed to monotonically decrease the quantization objective value, and outperforms existing methods in this category. We release the code at https://github.com/snu-mllab/GuidedQuant.

Key facts. Cited by 28 papers in the map (data through 7 Oct 2026). Builds on 44 of its 61 references that are in the map. First posted on arXiv 11 May 2025.

Builds on

44 of 61 references.

References · 44

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  5. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  6. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  7. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  8. 2016Pointer Sentinel Mixture Modelscited by 2,992
  9. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,818
  10. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,140
  11. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,114
  12. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,838
32 more
  1. 2015Optimizing Neural Networks with Kronecker-factored Approximate Curvaturecited by 776
  2. 2020Up or Down? Adaptive Rounding for Post-Training Quantizationcited by 600
  3. 2024QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMscited by 598
  4. 2019Importance Estimation for Neural Network Pruningcited by 531
  5. 2021BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstructioncited by 464
  6. 2024SpinQuant: LLM quantization with learned rotationscited by 446
  7. 2023QuIP: 2-Bit Quantization of Large Language Models With Guaranteescited by 397
  8. 20218-bit Optimizers via Block-wise Quantizationcited by 395
  9. 2024QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebookscited by 354
  10. 2022Optimal Brain Compression: A Framework for Accurate Post-Training Quantization and Pruningcited by 317
  11. 2023SqueezeLLM: Dense-and-Sparse Quantizationcited by 285
  12. 2022The case for 4-bit precision: k-bit Inference Scaling Lawscited by 247
  13. 2024Extreme Compression of Large Language Models via Additive Quantizationcited by 223
  14. 2022The Optimal BERT Surgeon: Scalable and Accurate Second-Order Pruning for Large Language Modelscited by 115
  15. 2019EigenDamage: Structured Pruning in the Kronecker-Factored Eigenbasiscited by 98
  16. 2022LUT-GEMM: Quantized Matrix Multiplication based on LUTs for Efficient Inference in Large-Scale Generative Language Modelscited by 98
  17. 2018Faster gaze prediction with dense networks and Fisher pruningcited by 92
  18. 2024AI and Memory Wallcited by 89
  19. 2024QTIP: Quantization with Trellises and Incoherence Processingcited by 86
  20. 2024GPTVQ: The Blessing of Dimensionality for LLM Quantizationcited by 78
  21. 2019Global Sparse Momentum SGD for Pruning Very Deep Neural Networkscited by 70
  22. 2024PV-Tuning: Beyond Straight-Through Estimation for Extreme LLM Compressioncited by 59
  23. 2021M-FAC: Efficient Matrix-Free Approximations of Second-Order Informationcited by 54
  24. 2024Any-Precision LLM: Low-Cost Deployment of Multiple, Different-Sized LLMscited by 50
  25. 2016Towards the Limit of Network Quantizationcited by 49
  26. 2023QuantEase: Optimization-based Quantization for Language Modelscited by 28
  27. 2023Fairness-Aware Structured Pruning in Transformerscited by 21
  28. 2024Pushing the Limits of Large Language Model Quantization via the Linearity Theoremcited by 20
  29. 2017Fast Exact k-Means, k-Medians and Bregman Divergence Clustering in 1Dcited by 13
  30. 2024SKIM: Any-bit Quantization Pushing The Limits of Post-Training Quantizationcited by 6
  31. 2024CDQuant: Greedy Coordinate Descent for Accurate LLM Quantizationcited by 4
  32. 2024Log-Time K-Means Clustering for 1D Data: Novel Approaches with Proof and Implementationcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 6; 2026: 22.

28 papers.

Sort:

By area:

  1. 2025Model-Preserving Adaptive Roundingcited by 20
  2. 2025Q-Palette: Fractional-Bit Quantizers Toward Optimal Bit Allocation for Efficient LLM Deploymentcited by 9
  3. 2025AnyBCQ: Hardware Efficient Flexible Binary-Coded Quantization for Multi-Precision LLMscited by 8
  4. 2026CoreQ: Learning-Free Mismatch Correction and Successive Rounding for Quantizationcited by 5
  5. 2026WaterSIC: Information-Theoretically (Near) Optimal Linear Layer Quantizationcited by 5
  6. 2026ScaleBITS: Scalable Bitwidth Search for Hardware-Aligned Mixed-Precision LLMscited by 3
  7. 2026MoBiE: Efficient Inference of Mixture of Binary Experts under Post-Training Quantizationcited by 2
  8. 2026Saliency-Aware Regularized Quantization Calibration for Large Language Modelscited by 2
  9. 2026KronQ: LLM Quantization via Kronecker-Factored Hessiancited by 2
  10. 2026SoftWater: Class-Aware Rate Allocation for Softmax Quantizationcited by 2
  11. 2025A Comprehensive Evaluation on Quantization Techniques for Large Language Modelscited by 1
  12. 2025P/D-Device: Disaggregated Large Language Model between Cloud and Devicescited by 1
16 more
  1. 2026NanoQuant: Efficient Sub-1-Bit Quantization of Large Language Modelscited by 1
  2. 2026Beyond Activation Alignment:The Alignment-Diversity Tradeoff in Task-Aware LLM Quantizationcited by 1
  3. 2026Why Does Post-Training Quantization Work?cited by 1
  4. 2025Discrete and Continuous Difference of Submodular Minimizationcited by 0
  5. 2026GlowQ: Group-Shared LOw-Rank Approximation for Quantized LLMscited by 0
  6. 2026MGVQ: Synergizing Multi-dimensional Sensitivity-Aware and Gradient-Hessian Fusion for Vector Quantizationcited by 0
  7. 2026Saturation Makes Quantization Error Additive: A Coverage Model with a Certificatecited by 0
  8. 2026Tied Trit-Planes: Constraining PTQTP to a Uniform Nine-Level Quantizer, with a Persistent Folded Format for Disk-Streamed Mixture-of-Experts Servingcited by 0
  9. 2026SchurQuant: Groupwise Discrete Optimization for Layer-Wise LLM Quantizationcited by 0
  10. 2026The Asymmetric Harms of LLM Compressioncited by 0
  11. 2026REAL-Q: E2E LLM Quantization via Dynamic Gradient Descentcited by 0
  12. 2026Predicting Quantization Price for Selecting PTQ Configurations Before Deploymentcited by 0
  13. 2026G2PTQ: Improving LLM Post-Training Quantization with Generalized Gradient Compensationcited by 0
  14. 2026Calibrate the Decisions That Change the Future: On-Policy Post-Training Quantization for Multimodal Large Language Modelscited by 0
  15. 2026XOR-Trellis: Ultra-Low-Complexity Dequantization and Curvature-Aware Hadamard-Free LLM Quantizationcited by 0
  16. 2026When Lower Reconstruction Loss Hurts: Distributionally Robust Refinement for Low-Bit LLM Quantizationcited by 0