mlsanityData to 7 Oct 2026

KronQ: LLM Quantization via Kronecker-Factored Hessian

arXiv 2607.07964 ↗First posted 8 Jul 2026cs.LGcited by 2

Not on the map. Why?

Abstract (arXiv)

Post-training quantization (PTQ) is a widely adopted technique for compressing large language models (LLMs) without retraining. Most existing second-order PTQ methods, including GPTQ, construct quantization objectives from input activation statistics, effectively assuming that all output channels contribute equally to the layer-wise reconstruction objective. We propose KronQ, a PTQ framework that challenges this assumption by introducing the gradient covariance into the quantization pipeline. Under the Kronecker-factored Hessian approximation, the quantization loss depends jointly on both the activation and gradient covariances, and KronQ exploits this at two complementary levels. (1) KronQ introduces bidirectional incoherence processing, extending the existing input-side random rotation to the output dimension using the gradient covariance, reducing weight magnitude variance across both input and output dimensions. (2) KronQ derives a new sensitivity metric for inter-layer mixed-precision allocation, driven by the gradient and activation Hessian traces. Notably, in the case of 2-bit weight-only quantization on LLaMA-3-70B, while GPTQ and GPTAQ diverge or produce degenerate quantizations (>2000 perplexity on WikiText-2), KronQ achieves 7.93 perplexity.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 45 of its 55 references that are in the map. First posted on arXiv 8 Jul 2026.

Builds on

45 of 55 references.

References · 45

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  5. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  6. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  7. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  8. 2023Mistral 7Bcited by 3,065
  9. 2016Pointer Sentinel Mixture Modelscited by 2,992
  10. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,818
  11. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
  12. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,140
33 more
  1. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,114
  2. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 1,906
  3. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,838
  4. 2025Gemma 3 Technical Reportcited by 1,686
  5. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,338
  6. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,234
  7. 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 798
  8. 2015Optimizing Neural Networks with Kronecker-factored Approximate Curvaturecited by 776
  9. 2024QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMscited by 598
  10. 2025Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAscited by 491
  11. 2024SpinQuant: LLM quantization with learned rotationscited by 446
  12. 2023OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Modelscited by 403
  13. 2023QuIP: 2-Bit Quantization of Large Language Models With Guaranteescited by 397
  14. 2024QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebookscited by 354
  15. 2022Optimal Brain Compression: A Framework for Accurate Post-Training Quantization and Pruningcited by 317
  16. 2021Pruning and Quantization for Deep Neural Network Acceleration: A Surveycited by 259
  17. 2019HAWQ-V2: Hessian Aware trace-Weighted Quantization of Neural Networkscited by 250
  18. 2019Limitations of the Empirical Fisher Approximation for Natural Gradient Descentcited by 213
  19. 2017Practical Gauss-Newton Optimisation for Deep Learningcited by 139
  20. 2024FlatQuant: Flatness Matters for LLM Quantizationcited by 116
  21. 2025OstQuant: Refining Large Language Model Quantization with Orthogonal and Scaling Transformations for Better Distribution Fittingcited by 94
  22. 2025Quantization Hurts Reasoning? An Empirical Study on Quantized Reasoning Modelscited by 68
  23. 2024SliM-LLM: Salience-Driven Mixed-Precision Quantization for Large Language Modelscited by 61
  24. 2025GPTAQ: Efficient Finetuning-Free Quantization for Asymmetric Calibrationcited by 54
  25. 2025Quantization Error Propagation: Revisiting Layer-Wise Post-Training Quantizationcited by 36
  26. 2025GuidedQuant: Large Language Model Quantization via Exploiting End Loss Guidancecited by 28
  27. 2025Model-Preserving Adaptive Roundingcited by 20
  28. 2025AMQ: Enabling AutoML for Mixed-precision Weight-Only Quantization of Large Language Modelscited by 15
  29. 2024BoA: Attention-aware Post-training Quantization without Backpropagationcited by 12
  30. 2025Benchmarking Post-Training Quantization in LLMs: Comprehensive Taxonomy, Unified Evaluation, and Comparative Analysiscited by 12
  31. 2024Channel-Wise Mixed-Precision Quantization for Large Language Modelscited by 11
  32. 2025Q-Palette: Fractional-Bit Quantizers Toward Optimal Bit Allocation for Efficient LLM Deploymentcited by 9
  33. 2024The Uniqueness of LLaMA3-70B Series with Per-Channel Quantizationcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026KBBQ: A Predictive Noise Law and the Limits of Spectrum Flattening in FP4 Quantizationcited by 0
  2. 2026SoloQ: Calibration-Free Quantization for Diffusion Language Modelscited by 0