# Optimize Weight Rounding via Signed Gradient Descent for the Quantization of LLMs

arXiv: https://arxiv.org/abs/2309.05516 · HTML: https://mlsanity.com/paper/2309.05516/

## Key facts

- Cited by 50 papers in the map (data through 14 Oct 2026).
- Builds on 51 of its 77 references that are in the map.
- First posted on arXiv 11 Sep 2023.

## Builds on (most cited first, 30 of 51)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge](https://mlsanity.com/paper/1803.05457/) (2018)
- [HellaSwag: Can a Machine Really Finish Your Sentence?](https://mlsanity.com/paper/1905.07830/) (2019)
- [OPT: Open Pre-trained Transformer Language Models](https://mlsanity.com/paper/2205.01068/) (2022)
- [QLoRA: Efficient Finetuning of Quantized LLMs](https://mlsanity.com/paper/2305.14314/) (2023)
- [Pointer Sentinel Mixture Models](https://mlsanity.com/paper/1609.07843/) (2016)
- [TruthfulQA: Measuring How Models Mimic Human Falsehoods](https://mlsanity.com/paper/2109.07958/) (2021)
- [PIQA: Reasoning about Physical Commonsense in Natural Language](https://mlsanity.com/paper/1911.11641/) (2019)
- [The Pile: An 800GB Dataset of Diverse Text for Language Modeling](https://mlsanity.com/paper/2101.00027/) (2020)
- [BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions](https://mlsanity.com/paper/1905.10044/) (2019)
- [Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answering](https://mlsanity.com/paper/1809.02789/) (2018)
- [Code Llama: Open Foundation Models for Code](https://mlsanity.com/paper/2308.12950/) (2023)
- [Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation](https://mlsanity.com/paper/1308.3432/) (2013)
- [BLOOM: A 176B-Parameter Open-Access Multilingual Language Model](https://mlsanity.com/paper/2211.05100/) (2022)
- [GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers](https://mlsanity.com/paper/2210.17323/) (2022)
- [SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models](https://mlsanity.com/paper/2211.10438/) (2022)
- [AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration](https://mlsanity.com/paper/2306.00978/) (2023)
- [The LAMBADA dataset: Word prediction requiring a broad discourse context](https://mlsanity.com/paper/1606.06031/) (2016)
- [LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale](https://mlsanity.com/paper/2208.07339/) (2022)
- [Learned Step Size Quantization](https://mlsanity.com/paper/1902.08153/) (2019)
- [Up or Down? Adaptive Rounding for Post-Training Quantization](https://mlsanity.com/paper/2004.10568/) (2020)
- [BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstruction](https://mlsanity.com/paper/2102.05426/) (2021)
- [HAQ: Hardware-Aware Automated Quantization with Mixed Precision](https://mlsanity.com/paper/1811.08886/) (2018)
- [OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Models](https://mlsanity.com/paper/2308.13137/) (2023)
- [Data-Free Quantization Through Weight Equalization and Bias Correction](https://mlsanity.com/paper/1906.04721/) (2019)

## Built on by (most cited first, 30 of 50)

- [Mobile Edge Intelligence for Large Language Models: A Contemporary Survey](https://mlsanity.com/paper/2407.18921/) (2024)
- [EfficientQAT: Efficient Quantization-Aware Training for Large Language Models](https://mlsanity.com/paper/2407.11062/) (2024)
- [ReLU2 Wins: Discovering Efficient Activation Functions for Sparse LLMs](https://mlsanity.com/paper/2402.03804/) (2024)
- [AffineQuant: Affine Transformation Quantization for Large Language Models](https://mlsanity.com/paper/2403.12544/) (2024)
- [QLLM: Accurate and Efficient Low-Bitwidth Quantization for Large Language Models](https://mlsanity.com/paper/2310.08041/) (2023)
- [Model Compression and Efficient Inference for Large Language Models: A Survey](https://mlsanity.com/paper/2402.09748/) (2024)
- [Baichuan-M2: Scaling Medical Capability with Large Verifier System](https://mlsanity.com/paper/2509.02208/) (2025)
- [Optimizing Large Language Model Training Using FP4 Quantization](https://mlsanity.com/paper/2501.17116/) (2025)
- [A Survey on Transformer Compression](https://mlsanity.com/paper/2402.05964/) (2024)
- [FP4 All the Way: Fully Quantized Training of LLMs](https://mlsanity.com/paper/2505.19115/) (2025)
- [SiDA-MoE: Sparsity-Inspired Data-Aware Serving for Efficient and Scalable Large Mixture-of-Experts Models](https://mlsanity.com/paper/2310.18859/) (2023)
- [Perturbation-Restrained Sequential Model Editing](https://mlsanity.com/paper/2405.16821/) (2024)
- [Qronos: Correcting the Past by Shaping the Future... in Post-Training Quantization](https://mlsanity.com/paper/2505.11695/) (2025)
- [Efficient LLM Inference on CPUs](https://mlsanity.com/paper/2311.00502/) (2023)
- [Oaken: Fast and Efficient LLM Serving with Online-Offline Hybrid KV Cache Quantization](https://mlsanity.com/paper/2503.18599/) (2025)
- [Multilingual Brain Surgeon: Large Language Models Can be Compressed Leaving No Language Behind](https://mlsanity.com/paper/2404.04748/) (2024)
- [Baichuan-M3: Modeling Clinical Inquiry for Reliable Medical Decision-Making](https://mlsanity.com/paper/2602.06570/) (2026)
- [Unifying Block-wise PTQ and Distillation-based QAT for Progressive Quantization toward 2-bit Instruction-Tuned LLMs](https://mlsanity.com/paper/2506.09104/) (2025)
- [Effective Quantization for Diffusion Models on CPUs](https://mlsanity.com/paper/2311.16133/) (2023)
- [Value-and-Structure Alignment for Routing-Consistent Quantization of Mixture-of-Experts Models](https://mlsanity.com/paper/2606.05688/) (2026)
- [LoTA-QAF: Lossless Ternary Adaptation for Quantization-Aware Fine-Tuning](https://mlsanity.com/paper/2505.18724/) (2025)
- [MoPEQ: Mixture of Mixed Precision Quantized Experts](https://mlsanity.com/paper/2509.02512/) (2025)
- [TesseraQ: Ultra Low-Bit LLM Post-Training Quantization with Block Reconstruction](https://mlsanity.com/paper/2410.19103/) (2024)
- [Qrazor: Reliable and Effortless 4-bit LLM Quantization by Significant Data Razoring](https://mlsanity.com/paper/2501.13331/) (2025)
- [High-Dimensional Learning Dynamics of Quantized Models with Straight-Through Estimator](https://mlsanity.com/paper/2510.10693/) (2025)
- [Beacon: Post-Training Quantization with Integrated Grid Selection](https://mlsanity.com/paper/2508.20293/) (2025)
- [Optimal Post-Training Quantization Scales and Where to Find Them](https://mlsanity.com/paper/2606.10890/) (2026)
- [EdgeRazor: A Lightweight Framework for Large Language Models via Mixed-Precision Quantization-Aware Distillation](https://mlsanity.com/paper/2605.04062/) (2026)
- [LFQ: Logit-aware Final-block Quantization for Boosting the Generation Quality of Low-Bit Quantized LLMs](https://mlsanity.com/paper/2605.29756/) (2026)
- [UniSVQ: 2-bit Unified Scalar-Vector Quantization](https://mlsanity.com/paper/2606.10520/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) have demonstrated exceptional proficiency in language-related tasks, but their deployment poses significant challenges due to substantial memory and storage requirements. Weight-only quantization has emerged as a promising solution, significantly reducing memory and storage needs without sacrificing too much performance. In this study, we introduce SignRound, a method that leverages signed gradient descent (SignSGD) to optimize rounding values and weight clipping in just 200 steps. SignRound integrates the advantages of Quantization-Aware Training (QAT) and Post-Training Quantization (PTQ), delivering exceptional results across 2 to 4 bits while minimizing tuning costs and avoiding additional inference overhead. For example, SignRound achieved absolute average accuracy improvements ranging from 6.91% to 33.22% at 2bits, as measured by the average zero-shot accuracy across 11 tasks. It also demonstrates strong generalization in recent models, achieving near-lossless 4-bit quantization in most scenarios. The source code is publicly available at https://github.com/intel/auto-round.