mlsanityData to 7 Oct 2026

Scaling Laws for Post Training Quantized Large Language Models

arXiv 2410.12119 ↗First posted 15 Oct 2024cs.LG cs.CLcited by 4

Not on the map. Why?

Abstract (arXiv)

Generalization abilities of well-trained large language models (LLMs) are known to scale predictably as a function of model size. In contrast to the existence of practical scaling laws governing pre-training, the quality of LLMs after post-training compression remains highly unpredictable, often requiring case-by-case validation in practice. In this work, we attempted to close this gap for post-training weight quantization of LLMs by conducting a systematic empirical study on multiple LLM families quantized to numerous low-precision tensor data types using popular weight quantization techniques. We identified key scaling factors pertaining to characteristics of the local loss landscape, based on which the performance of quantized LLMs can be reasonably well predicted by a statistical model.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 28 of its 35 references that are in the map. First posted on arXiv 15 Oct 2024.

Builds on

28 of 35 references.

References · 28

  1. 2020Scaling Laws for Neural Language Modelscited by 6,411
  2. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  3. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  4. 2016Pointer Sentinel Mixture Modelscited by 2,992
  5. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,838
  6. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,734
  7. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,338
  8. 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 798
  9. 2022ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformerscited by 517
  10. 2020Scaling Laws for Autoregressive Generative Modelingcited by 510
  11. 2023Scaling Data-Constrained Language Modelscited by 391
  12. 2021Explaining Neural Scaling Lawscited by 353
16 more
  1. 2019PyHessian: Neural Networks Through the Lens of the Hessiancited by 291
  2. 2022The case for 4-bit precision: k-bit Inference Scaling Lawscited by 247
  3. 2022Revisiting Neural Scaling Laws in Language and Visioncited by 129
  4. 2023OWQ: Outlier-Aware Weight Quantization for Efficient Fine-Tuning and Inference of Large Language Modelscited by 113
  5. 2024A Dynamical Model of Neural Scaling Lawscited by 105
  6. 2022LUT-GEMM: Quantized Matrix Multiplication based on LUTs for Efficient Inference in Large-Scale Generative Language Modelscited by 98
  7. 2019Loss Aware Post-training Quantizationcited by 94
  8. 2019The Difficulty of Training Sparse Neural Networkscited by 62
  9. 2024Unraveling the Mystery of Scaling Laws: Part Icited by 24
  10. 2023FineQuant: Unlocking Efficiency with Fine-Grained Weight-Only Quantization for LLMscited by 22
  11. 2023Jumping through Local Minima: Quantization in the Loss Landscape of Vision Transformerscited by 18
  12. 2024L4Q: Parameter Efficient Quantization-Aware Fine-Tuning on Large Language Modelscited by 15
  13. 2023Benchmarking the Reliability of Post-training Quantization: a Particular Focus on Worst-case Performancecited by 8
  14. 2024A Resource Model For Neural Scaling Lawcited by 8
  15. 2024eXmY: A Data Type and Technique for Arbitrary Bit Precision Quantizationcited by 7
  16. 2022Empirical Evaluation of Post-Training Quantization Methods for Language Taskscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 2.

4 papers.

  1. 2025Compressibility Measures Complexity: Minimum Description Length Meets Singular Learning Theorycited by 4
  2. 2026The Structure of Quantization Damage in LLMs: Why the Next Bit Should Be Spent Globallycited by 1
  3. 2025OptRot: Mitigating Weight Outliers via Data-Free Rotations for Post-Training Quantizationcited by 0
  4. 2026LLMOrbit: A Circular Taxonomy of Large Language Models -From Scaling Walls to Agentic AI Systemscited by 0