mlsanityData to 14 Oct 2026

HeRo-Q: A General Framework for Stable Low Bit Quantization via Hessian Conditioning

arXiv 2601.21626 ↗First posted 29 Jan 2026cs.LG cs.AIcited by 1

Not on the map. Why?

Abstract (arXiv)

Post Training Quantization (PTQ), a mainstream model compression technique, often leads to the paradoxical 'low error, high loss' phenomenon because it focuses solely on minimizing quantization error. The root cause lies in the Hessian matrix of the LLM loss landscape: a few high curvature directions are extremely sensitive to perturbations. To address this, we propose the Hessian Robust Quantization (HeRo Q) algorithm, which applies a lightweight, learnable rotation-compression matrix to the weight space prior to quantization. This joint framework reshapes the loss landscape by reducing the largest Hessian eigenvalue and reducing its max eigenvalue, thereby significantly enhancing robustness to quantization noise. HeRo-Q requires no architectural modifications, incurs negligible computational overhead, and integrates seamlessly into existing PTQ pipelines. Experiments on Llama and Qwen models show that HeRo Q consistently outperforms state of the art methods including GPTQ, AWQ, and SpinQuant not only achieving superior performance under standard W4A8 settings, but also excelling in the highly challenging W3A16 ultra low bit regime, where it boosts GSM8K accuracy on Llama3 8B to 70.15% and effectively avoids the logical collapse commonly seen in aggressive quantization.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 51 of its 62 references that are in the map. First posted on arXiv 29 Jan 2026.

Builds on

51 of 62 references.

References · 51

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  4. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  5. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  6. 2016Layer Normalizationcited by 5,669
  7. 2024Qwen2.5 Technical Reportcited by 4,942
  8. 2020Training data-efficient image transformers & distillation through attentioncited by 4,586
  9. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,100
  10. 2023Qwen Technical Reportcited by 3,344
  11. 2016Pointer Sentinel Mixture Modelscited by 3,096
  12. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
38 more
  1. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
  2. 2019Root Mean Square Layer Normalizationcited by 1,480
  3. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
  4. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,298
  5. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 906
  6. 2024DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Modelscited by 867
  7. 2024QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMscited by 634
  8. 2020Up or Down? Adaptive Rounding for Post-Training Quantizationcited by 621
  9. 2024SpinQuant: LLM quantization with learned rotationscited by 471
  10. 2018HAQ: Hardware-Aware Automated Quantization with Mixed Precisioncited by 443
  11. 2023OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Modelscited by 424
  12. 2023QuIP: 2-Bit Quantization of Large Language Models With Guaranteescited by 414
  13. 2017Empirical Analysis of the Hessian of Over-Parametrized Neural Networkscited by 290
  14. 2020ZeroQ: A Novel Zero Shot Quantization Frameworkcited by 269
  15. 2019HAWQ-V2: Hessian Aware trace-Weighted Quantization of Neural Networkscited by 261
  16. 2021PTQ4ViT: Post-training quantization for vision transformers with twin uniform quantizationcited by 179
  17. 2021FQ-ViT: Post-Training Quantization for Fully Quantized Vision Transformercited by 151
  18. 2022RepQ-ViT: Scale Reparameterization for Post-Training Quantization of Vision Transformerscited by 137
  19. 2024FlatQuant: Flatness Matters for LLM Quantizationcited by 131
  20. 2018Mixed Precision Quantization of ConvNets via Differentiable Neural Architecture Searchcited by 125
  21. 2024Why Transformers Need Adam: A Hessian Perspectivecited by 124
  22. 2023RPTQ: Reorder-based Post-training Quantization for Large Language Modelscited by 116
  23. 2025OstQuant: Refining Large Language Model Quantization with Orthogonal and Scaling Transformations for Better Distribution Fittingcited by 97
  24. 2020Efficient Riemannian Optimization on the Stiefel Manifold via the Cayley Transformcited by 80
  25. 2023Towards Accurate Post-Training Quantization for Vision Transformercited by 71
  26. 2019AutoQ: Automated Kernel-Wise Neural Network Quantizationcited by 63
  27. 2021Towards Mixed-Precision Quantization of Neural Networks via Constrained Optimizationcited by 61
  28. 2024VPTQ: Extreme Low-bit Vector Post-Training Quantization for Large Language Modelscited by 57
  29. 2022Mixed-Precision Neural Network Quantization via Learned Layer-wise Importancecited by 54
  30. 2024MBQ: Modality-Balanced Quantization for Large Vision-Language Modelscited by 45
  31. 2020A Deeper Look at the Hessian Eigenspectrum of Deep Neural Networks and its Applications to Regularizationcited by 44
  32. 2025MoEQuant: Enhancing Quantization for Mixture-of-Experts Large Language Models via Expert-Balanced Sampling and Affinity Guidancecited by 34
  33. 2025Systematic Outliers in Large Language Modelscited by 33
  34. 2025Unifying Uniform and Binary-coding Quantization for Accurate Compression of Large Language Modelscited by 5
  35. 2024Compression for Better: A General and Stable Lossless Compression Frameworkcited by 4
  36. 2025A General Error-Theoretical Analysis Framework for Constructing Compression Strategiescited by 3
  37. 2025MoQE: Improve Quantization Model performance via Mixture of Quantization Expertscited by 3
  38. 2025CALM: A CKA-Guided Adaptive Layer-Wise Modularization Framework for LLM Quantizationcited by 3

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Robust Ultra Low-Bit Post-Training Quantization via Stable Diagonal Curvature Estimatecited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2026HAS-VQ: Hessian-Adaptive Sparse Vector Quantization for High-Fidelity LLM Compression
  2. 2021HERO: Hessian-Enhanced Robust Optimization for Unifying and Improving Generalization and Quantization Performance
  3. 2026StableQAT: Stable Quantization-Aware Training at Ultra-Low Bitwidths
  4. 2024QQQ: Quality Quattuor-Bit Quantization for Large Language Models
  5. 2025ParetoQ: Improving Scaling Laws in Extremely Low-bit LLM Quantization
  6. 2026CAT-Q: Cost-efficient and Accurate Ternary Quantization for LLMs
  7. 2026When Lower Reconstruction Loss Hurts: Distributionally Robust Refinement for Low-Bit LLM Quantization
  8. 2026Few Bits, One Law: Toward W2A4KV2