mlsanityData to 14 Oct 2026

G2PTQ: Improving LLM Post-Training Quantization with Generalized Gradient Compensation

arXiv 2609.31009 ↗First posted 25 Sep 2026cs.CL cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Post-training quantization (PTQ) is a practical approach to reducing the memory and computational footprint of large language models (LLMs) without retraining. GPTQ-based methods have become the de facto standard, yet they suffer from two complementary limitations. Methods with local, layer-wise objectives lack global supervision; while methods with global objectives fix their Hessian estimates at the start and ignore first-order gradients, so their guidance grows stale as quantization proceeds. This paper presents G2PTQ, a unified PTQ framework with Generalized Gradient Compensation that integrates both first- and second-order information under a globally supervised, block-wise optimization objective. By refreshing gradient and Hessian estimates before quantizing each Transformer block, G2PTQ avoids the staleness of prior global methods. Furthermore, to stabilize the exact first-order compensation, we introduce a trust-region scaling mechanism that dynamically bounds the gradient step to prevent exploding weight updates. Finally, we derive efficient implementations for block-wise Hessian approximation and exact gradient compensation. Experimental results on various model families and bit-widths demonstrate that G2PTQ enables better alignment with the full-precision model, outperforming state-of-the-art baselines. Code is available at: https://github.com/G2PTQ/G2PTQ.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 45 of its 52 references that are in the map. First posted on arXiv 25 Sep 2026.

Builds on

45 of 52 references.

References · 45

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2025Qwen3 Technical Reportcited by 8,820
  3. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  4. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,100
  5. 2016Pointer Sentinel Mixture Modelscited by 3,096
  6. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,883
  7. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
  8. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,186
  9. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 2,004
  10. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
  11. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,298
  12. 2016The LAMBADA dataset: Word prediction requiring a broad discourse contextcited by 992
33 more
  1. 2023Enhancing Chat Language Models by Scaling High-quality Instructional Conversationscited by 839
  2. 2023C-Eval: A Multi-Level Multi-Discipline Chinese Evaluation Suite for Foundation Modelscited by 708
  3. 2024QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMscited by 634
  4. 2021A White Paper on Neural Network Quantizationcited by 472
  5. 2024SpinQuant: LLM quantization with learned rotationscited by 471
  6. 2023QuIP: 2-Bit Quantization of Large Language Models With Guaranteescited by 414
  7. 2023LLM-QAT: Data-Free Quantization Aware Training for Large Language Modelscited by 324
  8. 2022Optimal Brain Compression: A Framework for Accurate Post-Training Quantization and Pruningcited by 322
  9. 2024Massive Activations in Large Language Modelscited by 294
  10. 2025Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-Freecited by 283
  11. 2024Adam-mini: Use Fewer Learning Rates To Gain Morecited by 137
  12. 2026Beyond Benchmarks: MathArena as an Evaluation Platform for Mathematics with LLMscited by 137
  13. 2024FlatQuant: Flatness Matters for LLM Quantizationcited by 131
  14. 2025OstQuant: Refining Large Language Model Quantization with Orthogonal and Scaling Transformations for Better Distribution Fittingcited by 97
  15. 2024BitDistiller: Unleashing the Potential of Sub-4-Bit LLMs via Self-Distillationcited by 84
  16. 2025Quantization Hurts Reasoning? An Empirical Study on Quantized Reasoning Modelscited by 73
  17. 2025GPTAQ: Efficient Finetuning-Free Quantization for Asymmetric Calibrationcited by 60
  18. 2024IntactKV: Improving Large Language Model Quantization by Keeping Pivot Tokens Intactcited by 52
  19. 2025RotateKV: Accurate and Robust 2-Bit KV Cache Quantization for LLMs via Outlier-Aware Adaptive Rotationscited by 46
  20. 2026On the Design of Qwen3.8-Next Architecture: Evaluation, Efficiency, and Training Stabilitycited by 35
  21. 2025GuidedQuant: Large Language Model Quantization via Exploiting End Loss Guidancecited by 32
  22. 2024Accuracy is Not All You Needcited by 31
  23. 2025Model-Preserving Adaptive Roundingcited by 23
  24. 2025FPTQuant: Function-Preserving Transforms for LLM Quantizationcited by 16
  25. 2024BoA: Attention-aware Post-training Quantization without Backpropagationcited by 12
  26. 2025Identifying Sensitive Weights via Post-quantization Integralcited by 11
  27. 2026What Makes Low-Bit Quantization-Aware Training Work for Reasoning LLMs? A Systematic Studycited by 10
  28. 2025First-Order Error Matters: Accurate Compensation for Quantized Large Language Modelscited by 8
  29. 2024OAC: Output-adaptive Calibration for Accurate Post-training Quantizationcited by 7
  30. 2026Statistically-Lossless Quantization of Large Language Modelscited by 6
  31. 2025DiscQuant: A Quantization Method for Neural Networks Inspired by Discrepancy Theorycited by 3
  32. 2026TurboBoA: Faster and Exact Attention-aware Quantization without Backpropagationcited by 3
  33. 2026H-Scale: Hessian-Guided Scale Refinement for NVFP4 Sub-Byte LLM Inferencecited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026REAL-Q: E2E LLM Quantization via Dynamic Gradient Descent
  2. 2023ZeroQuant-V2: Exploring Post-training Quantization in LLMs from Comprehensive Study to Low Rank Compensation
  3. 2025PTQ1.61: Push the Real Limit of Extremely Low-Bit Post-Training Quantization Methods for Large Language Models
  4. 2025Layer-Wise High-Impact Parameter Ratio Optimization in Post-Training Quantization for Large Language Models
  5. 2024LRQ: Optimizing Post-Training Quantization for Large Language Models by Learning Low-Rank Weight-Scaling Matrices
  6. 2025Provable Post-Training Quantization: Theoretical Analysis of OPTQ and Qronos
  7. 2026KronQ: LLM Quantization via Kronecker-Factored Hessian
  8. 2026SEPTQ: A Simple and Effective Post-Training Quantization Paradigm for Large Language Models