mlsanityData to 7 Oct 2026

Predicting Quantization Price for Selecting PTQ Configurations Before Deployment

arXiv 2609.28270 ↗First posted 23 Sep 2026cs.CL cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Weight-space post-training quantization (PTQ) must choose finite formats, granularities, quantizer families, transformations, and bits before the completed quantized model reveals its output-distribution drift. Existing PTQ methods predict important pieces of this degradation, including reconstruction error, Hessian sensitivity, transformation effects, and downstream loss, but these pieces are usually scored after fixing the quantization geometry or inside separate configuration families. We formulate weight-space PTQ as pre-deployment configuration selection using priced layer-output error. Each admissible layer configuration is treated as an error generator with a deployment cost, which induces a layer-output error covariance boldsymbolSigmal(αl), and the full-precision model prices that covariance by downstream curvature, widehatρl(αl)=frac12operatornameTrleft(widehatHl,widehatboldsymbolSigmal(αl)right). The price follows from full-precision-to-quantized forward KL, whose first-order term cancels at the reference model. It turns reconstruction and diagonal scores into reduced proxies that drop price factors, while finite formats, codebooks, granularities, and equivalent transformations become comparable candidates through the covariances they induce and the costs they pay. A trace reduction then yields a calibration-time price table and a budgeted price-guided selector, making fixed-geometry bit allocation a special case rather than the organizing problem.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 51 of its 55 references that are in the map. First posted on arXiv 23 Sep 2026.

Builds on

51 of 55 references.

References · 51

  1. 2025Qwen3 Technical Reportcited by 8,195
  2. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  3. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  4. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,818
  5. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,140
  6. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,838
  7. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,338
  8. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 881
  9. 2024KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cachecited by 604
  10. 2020Up or Down? Adaptive Rounding for Post-Training Quantizationcited by 600
  11. 2024QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMscited by 598
  12. 2024KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantizationcited by 571
38 more
  1. 2021BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstructioncited by 464
  2. 2024SpinQuant: LLM quantization with learned rotationscited by 446
  3. 2023OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Modelscited by 403
  4. 2023QuIP: 2-Bit Quantization of Large Language Models With Guaranteescited by 397
  5. 2019HAWQ: Hessian AWare Quantization of Neural Networks with Mixed-Precisioncited by 387
  6. 2018WiC: the Word-in-Context Dataset for Evaluating Context-Sensitive Meaning Representationscited by 370
  7. 2023SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compressioncited by 359
  8. 2024QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebookscited by 354
  9. 2022Optimal Brain Compression: A Framework for Accurate Post-Training Quantization and Pruningcited by 317
  10. 2023SqueezeLLM: Dense-and-Sparse Quantizationcited by 285
  11. 2023Atom: Low-bit Quantization for Efficient and Accurate LLM Servingcited by 255
  12. 2019HAWQ-V2: Hessian Aware trace-Weighted Quantization of Neural Networkscited by 250
  13. 2024Extreme Compression of Large Language Models via Additive Quantizationcited by 223
  14. 2024QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Servingcited by 200
  15. 2022Outlier Suppression: Pushing the Limit of Low-bit Transformer Language Modelscited by 176
  16. 2024FlatQuant: Flatness Matters for LLM Quantizationcited by 116
  17. 2024QTIP: Quantization with Trellises and Incoherence Processingcited by 86
  18. 2024GPTVQ: The Blessing of Dimensionality for LLM Quantizationcited by 78
  19. 2024AffineQuant: Affine Transformation Quantization for Large Language Modelscited by 64
  20. 2024SliM-LLM: Salience-Driven Mixed-Precision Quantization for Large Language Modelscited by 61
  21. 2024VPTQ: Extreme Low-bit Vector Post-Training Quantization for Large Language Modelscited by 53
  22. 2021NASI: Label- and Data-agnostic Neural Architecture Search at Initializationcited by 28
  23. 2025GuidedQuant: Large Language Model Quantization via Exploiting End Loss Guidancecited by 28
  24. 2024QERA: an Analytical Framework for Quantization Error Reconstructioncited by 21
  25. 2024Pushing the Limits of Large Language Model Quantization via the Linearity Theoremcited by 20
  26. 2025Model-Preserving Adaptive Roundingcited by 20
  27. 2022Unifying and Boosting Gradient-Based Training-Free Neural Architecture Searchcited by 17
  28. 2025AMQ: Enabling AutoML for Mixed-precision Weight-Only Quantization of Large Language Modelscited by 15
  29. 2024QQQ: Quality Quattuor-Bit Quantization for Large Language Modelscited by 14
  30. 2025Provable Post-Training Quantization: Theoretical Analysis of OPTQ and Qronoscited by 13
  31. 2024LRQ: Optimizing Post-Training Quantization for Large Language Models by Learning Low-Rank Weight-Scaling Matricescited by 11
  32. 2024Post Training Quantization of Large Language Models with Microscaling Formatscited by 10
  33. 2025Q-Palette: Fractional-Bit Quantizers Toward Optimal Bit Allocation for Efficient LLM Deploymentcited by 9
  34. 2024ASER: Activation Smoothing and Error Reconstruction for Large Language Model Quantizationcited by 8
  35. 2026Benchmarking Post-Training Quantization of Large Language Models under Microscaling Floating Point Formatscited by 8
  36. 2025PTQ1.61: Push the Real Limit of Extremely Low-Bit Post-Training Quantization Methods for Large Language Modelscited by 7
  37. 2025CALM: A CKA-Guided Adaptive Layer-Wise Modularization Framework for LLM Quantizationcited by 3
  38. 2026Activation Sensitivity as a Unifying Principle for Post-Training Quantizationcited by 1

50 most cited of 51 All 51 (JSON)

Built on by

Not cited yet.