mlsanityData to 14 Oct 2026

Optimize Weight Rounding via Signed Gradient Descent for the Quantization of LLMs

arXiv 2309.05516 ↗First posted 11 Sep 2023cs.CL cs.AI cs.LGcited by 50

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) have demonstrated exceptional proficiency in language-related tasks, but their deployment poses significant challenges due to substantial memory and storage requirements. Weight-only quantization has emerged as a promising solution, significantly reducing memory and storage needs without sacrificing too much performance. In this study, we introduce SignRound, a method that leverages signed gradient descent (SignSGD) to optimize rounding values and weight clipping in just 200 steps. SignRound integrates the advantages of Quantization-Aware Training (QAT) and Post-Training Quantization (PTQ), delivering exceptional results across 2 to 4 bits while minimizing tuning costs and avoiding additional inference overhead. For example, SignRound achieved absolute average accuracy improvements ranging from 6.91% to 33.22% at 2bits, as measured by the average zero-shot accuracy across 11 tasks. It also demonstrates strong generalization in recent models, achieving near-lossless 4-bit quantization in most scenarios. The source code is publicly available at https://github.com/intel/auto-round.

Key facts. Cited by 50 papers in the map (data through 14 Oct 2026). Builds on 51 of its 77 references that are in the map. First posted on arXiv 11 Sep 2023.

Builds on

51 of 77 references.

References · 51

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  5. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  6. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  7. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  8. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,100
  9. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  10. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  11. 2016Pointer Sentinel Mixture Modelscited by 3,096
  12. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
38 more
  1. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,883
  2. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,411
  3. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,183
  4. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,164
  5. 2023Code Llama: Open Foundation Models for Codecited by 2,139
  6. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,079
  7. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,984
  8. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
  9. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
  10. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,298
  11. 2016The LAMBADA dataset: Word prediction requiring a broad discourse contextcited by 992
  12. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 906
  13. 2019Learned Step Size Quantizationcited by 628
  14. 2020Up or Down? Adaptive Rounding for Post-Training Quantizationcited by 621
  15. 2021BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstructioncited by 479
  16. 2018HAQ: Hardware-Aware Automated Quantization with Mixed Precisioncited by 443
  17. 2023OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Modelscited by 424
  18. 2019Data-Free Quantization Through Weight Equalization and Bias Correctioncited by 336
  19. 2023LLM-QAT: Data-Free Quantization Aware Training for Large Language Modelscited by 324
  20. 2022Optimal Brain Compression: A Framework for Accurate Post-Training Quantization and Pruningcited by 322
  21. 2023SqueezeLLM: Dense-and-Sparse Quantizationcited by 294
  22. 2021Post-Training Quantization for Vision Transformercited by 278
  23. 2022Outlier Suppression: Pushing the Limit of Low-bit Transformer Language Modelscited by 181
  24. 2022QDrop: Randomly Dropping Quantization for Extremely Low-bit Post-Training Quantizationcited by 180
  25. 2023Quantizable Transformers: Removing Outliers by Helping Attention Heads Do Nothingcited by 151
  26. 2023RPTQ: Reorder-based Post-training Quantization for Large Language Modelscited by 116
  27. 2023Memory-Efficient Fine-Tuning of Compressed Large Language Models via sub-4-bit Integer Quantizationcited by 100
  28. 2023ZeroQuant-V2: Exploring Post-training Quantization in LLMs from Comprehensive Study to Low Rank Compensationcited by 79
  29. 2023Outlier Suppression+: Accurate quantization of large language models by equivalent and optimal shifting and scalingcited by 61
  30. 2023Oscillation-free Quantization for Low-bit Vision Transformerscited by 57
  31. 2019Stochastic Sign Descent Methods: New Algorithms and Better Theorycited by 49
  32. 2023FlexRound: Learnable Rounding based on Element-wise Division for Post-Training Quantizationcited by 48
  33. 2020The Geometry of Sign Gradient Descentcited by 41
  34. 2023FineQuant: Unlocking Efficiency with Fine-Grained Weight-Only Quantization for LLMscited by 22
  35. 2019Effective Training of Convolutional Neural Networks with Low-bitwidth Weights and Activationscited by 18
  36. 2021Cluster-Promoting Quantization with Bit-Drop for Minimizing Network Quantization Losscited by 12
  37. 2022SPIQ: Data-Free Per-Channel Static Input Quantizationcited by 12
  38. 2023NUPES : Non-Uniform Post-Training Quantization via Power Exponent Searchcited by 8

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 4; 2024: 10; 2025: 17; 2026: 19.

50 papers.

Sort:

By area:

  1. 2024Mobile Edge Intelligence for Large Language Models: A Contemporary Surveycited by 128
  2. 2024EfficientQAT: Efficient Quantization-Aware Training for Large Language Modelscited by 127
  3. 2024ReLU2 Wins: Discovering Efficient Activation Functions for Sparse LLMscited by 67
  4. 2024AffineQuant: Affine Transformation Quantization for Large Language Modelscited by 67
  5. 2023QLLM: Accurate and Efficient Low-Bitwidth Quantization for Large Language Modelscited by 64
  6. 2024Model Compression and Efficient Inference for Large Language Models: A Surveycited by 64
  7. 2025Baichuan-M2: Scaling Medical Capability with Large Verifier Systemcited by 59
  8. 2025Optimizing Large Language Model Training Using FP4 Quantizationcited by 56
  9. 2024A Survey on Transformer Compressioncited by 46
  10. 2025FP4 All the Way: Fully Quantized Training of LLMscited by 40
  11. 2023SiDA-MoE: Sparsity-Inspired Data-Aware Serving for Efficient and Scalable Large Mixture-of-Experts Modelscited by 38
  12. 2024Perturbation-Restrained Sequential Model Editingcited by 28
38 more
  1. 2025Qronos: Correcting the Past by Shaping the Future... in Post-Training Quantizationcited by 28
  2. 2023Efficient LLM Inference on CPUscited by 24
  3. 2025Oaken: Fast and Efficient LLM Serving with Online-Offline Hybrid KV Cache Quantizationcited by 14
  4. 2024Multilingual Brain Surgeon: Large Language Models Can be Compressed Leaving No Language Behindcited by 11
  5. 2026Baichuan-M3: Modeling Clinical Inquiry for Reliable Medical Decision-Makingcited by 11
  6. 2025Unifying Block-wise PTQ and Distillation-based QAT for Progressive Quantization toward 2-bit Instruction-Tuned LLMscited by 8
  7. 2023Effective Quantization for Diffusion Models on CPUscited by 7
  8. 2026Value-and-Structure Alignment for Routing-Consistent Quantization of Mixture-of-Experts Modelscited by 6
  9. 2025LoTA-QAF: Lossless Ternary Adaptation for Quantization-Aware Fine-Tuningcited by 5
  10. 2025MoPEQ: Mixture of Mixed Precision Quantized Expertscited by 5
  11. 2024TesseraQ: Ultra Low-Bit LLM Post-Training Quantization with Block Reconstructioncited by 4
  12. 2025Qrazor: Reliable and Effortless 4-bit LLM Quantization by Significant Data Razoringcited by 4
  13. 2025High-Dimensional Learning Dynamics of Quantized Models with Straight-Through Estimatorcited by 4
  14. 2025Beacon: Post-Training Quantization with Integrated Grid Selectioncited by 3
  15. 2026Optimal Post-Training Quantization Scales and Where to Find Themcited by 3
  16. 2026EdgeRazor: A Lightweight Framework for Large Language Models via Mixed-Precision Quantization-Aware Distillationcited by 2
  17. 2026LFQ: Logit-aware Final-block Quantization for Boosting the Generation Quality of Low-Bit Quantized LLMscited by 2
  18. 2026UniSVQ: 2-bit Unified Scalar-Vector Quantizationcited by 2
  19. 2024Interactions Across Blocks in Post-Training Quantization of Large Language Modelscited by 1
  20. 2025Biases in Edge Language Models: Detection, Analysis, and Mitigationcited by 1
  21. 2025AIvaluateXR: An Evaluation Framework for on-Device AI in XR with Benchmarking Resultscited by 1
  22. 2025Regularization-based Framework for Quantization-, Fault- and Variability-Aware Trainingcited by 1
  23. 2025Dual Precision Quantization for Efficient and Accurate Deep Neural Networks Inferencecited by 1
  24. 2025The Uneven Impact of Post-Training Quantization in Machine Translationcited by 1
  25. 2026DAQ: Delta-Aware Quantization for Post-Training LLM Weight Compressioncited by 1
  26. 2026The Banach-Butterfly Invariant: Influence-Adaptive Walsh Geometry for Ternary Polynomial Threshold Functionscited by 1
  27. 2026Influence-Inspired Spectral Rotations for Extreme Low-Bit LLM Quantizationcited by 1
  28. 2026QAM-W: Joint 2D Codebook Quantization for LLM Weights via Hadamard Rotation and Activation-Aware Scalingcited by 1
  29. 2026GRINQH: Graded Input-based Quantization Hierarchy for Efficient LLM Generationcited by 1
  30. 2025SignRoundV2: Toward Closing the Performance Gap in Extremely Low-Bit Post-Training Quantization for LLMscited by 0
  31. 2026SleepVLM: A Rule-Grounded Vision-Language Model for Auditable Sleep Stagingcited by 0
  32. 2026XFP: Quality-Targeted Adaptive Codebook Quantization with Sparse Outlier Separation for LLM Inferencecited by 0
  33. 2026WINDQuant: Weight-Informed Neural Decision-Making for Global Mixed-Precision LLM Quantizationcited by 0
  34. 2026ReRound: Reconstructive Rounding to Resolve Midpoint Ambiguity in Calibration-Free LLM Quantizationcited by 0
  35. 2026Train Where the Quantized Model Goes: On-Policy Distillation for Low-Bit Reasoningcited by 0
  36. 2026IMC-CLINIC: Coupled Loss-Informed Newton Iterations for Clipping in Analog In-Memory Computingcited by 0
  37. 2026JARQ: Joint Alternating Refinement for Quantizationcited by 0
  38. 2026FidelityQ: Preserving Predictive Fidelity in Ultra-Low-Bit Language Model Quantizationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Revisiting Adaptive Rounding with Vectorized Reparameterization for LLM Quantization
  2. 2026ScaleBITS: Scalable Bitwidth Search for Hardware-Aligned Mixed-Precision LLMs
  3. 2024Direct Quantized Training of Language Models with Stochastic Rounding
  4. 2026D2Quant: Accurate Low-bit Post-Training Weight Quantization for LLMs
  5. 2025Tequila: Trapping-free Ternary Quantization for Large Language Models
  6. 2025Achieving binary weight and activation for LLMs using Post-Training Quantization
  7. 2025Rethinking 1-bit Optimization Leveraging Pre-trained Large Language Models
  8. 2024CDQuant: Greedy Coordinate Descent for Accurate LLM Quantization