# Scaling Laws for Post Training Quantized Large Language Models

arXiv: https://arxiv.org/abs/2410.12119 · HTML: https://mlsanity.com/paper/2410.12119/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 35 references that are in the map.
- First posted on arXiv 15 Oct 2024.

## Builds on (most cited first, 28 of 28)

- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [OPT: Open Pre-trained Transformer Language Models](https://mlsanity.com/paper/2205.01068/) (2022)
- [QLoRA: Efficient Finetuning of Quantized LLMs](https://mlsanity.com/paper/2305.14314/) (2023)
- [Pointer Sentinel Mixture Models](https://mlsanity.com/paper/1609.07843/) (2016)
- [GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers](https://mlsanity.com/paper/2210.17323/) (2022)
- [Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling](https://mlsanity.com/paper/2304.01373/) (2023)
- [SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models](https://mlsanity.com/paper/2211.10438/) (2022)
- [A Survey of Quantization Methods for Efficient Neural Network Inference](https://mlsanity.com/paper/2103.13630/) (2021)
- [ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformers](https://mlsanity.com/paper/2206.01861/) (2022)
- [Scaling Laws for Autoregressive Generative Modeling](https://mlsanity.com/paper/2010.14701/) (2020)
- [Scaling Data-Constrained Language Models](https://mlsanity.com/paper/2305.16264/) (2023)
- [Explaining Neural Scaling Laws](https://mlsanity.com/paper/2102.06701/) (2021)
- [PyHessian: Neural Networks Through the Lens of the Hessian](https://mlsanity.com/paper/1912.07145/) (2019)
- [The case for 4-bit precision: k-bit Inference Scaling Laws](https://mlsanity.com/paper/2212.09720/) (2022)
- [Revisiting Neural Scaling Laws in Language and Vision](https://mlsanity.com/paper/2209.06640/) (2022)
- [OWQ: Outlier-Aware Weight Quantization for Efficient Fine-Tuning and Inference of Large Language Models](https://mlsanity.com/paper/2306.02272/) (2023)
- [A Dynamical Model of Neural Scaling Laws](https://mlsanity.com/paper/2402.01092/) (2024)
- [LUT-GEMM: Quantized Matrix Multiplication based on LUTs for Efficient Inference in Large-Scale Generative Language Models](https://mlsanity.com/paper/2206.09557/) (2022)
- [Loss Aware Post-training Quantization](https://mlsanity.com/paper/1911.07190/) (2019)
- [The Difficulty of Training Sparse Neural Networks](https://mlsanity.com/paper/1906.10732/) (2019)
- [Unraveling the Mystery of Scaling Laws: Part I](https://mlsanity.com/paper/2403.06563/) (2024)
- [FineQuant: Unlocking Efficiency with Fine-Grained Weight-Only Quantization for LLMs](https://mlsanity.com/paper/2308.09723/) (2023)
- [Jumping through Local Minima: Quantization in the Loss Landscape of Vision Transformers](https://mlsanity.com/paper/2308.10814/) (2023)
- [L4Q: Parameter Efficient Quantization-Aware Fine-Tuning on Large Language Models](https://mlsanity.com/paper/2402.04902/) (2024)
- [Benchmarking the Reliability of Post-training Quantization: a Particular Focus on Worst-case Performance](https://mlsanity.com/paper/2303.13003/) (2023)
- [A Resource Model For Neural Scaling Law](https://mlsanity.com/paper/2402.05164/) (2024)
- [eXmY: A Data Type and Technique for Arbitrary Bit Precision Quantization](https://mlsanity.com/paper/2405.13938/) (2024)
- [Empirical Evaluation of Post-Training Quantization Methods for Language Tasks](https://mlsanity.com/paper/2210.16621/) (2022)

## Built on by (most cited first, 4 of 4)

- [Compressibility Measures Complexity: Minimum Description Length Meets Singular Learning Theory](https://mlsanity.com/paper/2510.12077/) (2025)
- [OptRot: Mitigating Weight Outliers via Data-Free Rotations for Post-Training Quantization](https://mlsanity.com/paper/2512.24124/) (2025)
- [The Structure of Quantization Damage in LLMs: Why the Next Bit Should Be Spent Globally](https://mlsanity.com/paper/2609.01587/) (2026)
- [LLMOrbit: A Circular Taxonomy of Large Language Models -From Scaling Walls to Agentic AI Systems](https://mlsanity.com/paper/2601.14053/) (2026)

## Abstract (arXiv)

Generalization abilities of well-trained large language models (LLMs) are known to scale predictably as a function of model size. In contrast to the existence of practical scaling laws governing pre-training, the quality of LLMs after post-training compression remains highly unpredictable, often requiring case-by-case validation in practice. In this work, we attempted to close this gap for post-training weight quantization of LLMs by conducting a systematic empirical study on multiple LLM families quantized to numerous low-precision tensor data types using popular weight quantization techniques. We identified key scaling factors pertaining to characteristics of the local loss landscape, based on which the performance of quantized LLMs can be reasonably well predicted by a statistical model.