# Improving Post Training Neural Quantization: Layer-wise Calibration and Integer Programming

arXiv: https://arxiv.org/abs/2006.10518 · HTML: https://mlsanity.com/paper/2006.10518/

## Key facts

- Cited by 92 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Jun 2020.

## Built on by (most cited first, 30 of 92)

- [GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers](https://mlsanity.com/paper/2210.17323/) (2022)
- [A Survey of Quantization Methods for Efficient Neural Network Inference](https://mlsanity.com/paper/2103.13630/) (2021)
- [BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstruction](https://mlsanity.com/paper/2102.05426/) (2021)
- [Post-training Quantization on Diffusion Models](https://mlsanity.com/paper/2211.15736/) (2022)
- [PTQD: Accurate Post-Training Quantization for Diffusion Models](https://mlsanity.com/paper/2305.10657/) (2023)
- [A Fast Post-Training Pruning Framework for Transformers](https://mlsanity.com/paper/2204.09656/) (2022)
- [Quantizable Transformers: Removing Outliers by Helping Attention Heads Do Nothing](https://mlsanity.com/paper/2306.12929/) (2023)
- [Accelerated Sparse Neural Training: A Provable and Efficient Method to Find N:M Transposable Masks](https://mlsanity.com/paper/2102.08124/) (2021)
- [TFMQ-DM: Temporal Feature Maintenance Quantization for Diffusion Models](https://mlsanity.com/paper/2311.16503/) (2023)
- [QLLM: Accurate and Efficient Low-Bitwidth Quantization for Large Language Models](https://mlsanity.com/paper/2310.08041/) (2023)
- [GPTAQ: Efficient Finetuning-Free Quantization for Asymmetric Calibration](https://mlsanity.com/paper/2504.02692/) (2025)
- [SQuant: On-the-Fly Data-Free Quantization via Diagonal Hessian Approximation](https://mlsanity.com/paper/2202.07471/) (2022)
- [Intriguing Properties of Quantization at Scale](https://mlsanity.com/paper/2305.19268/) (2023)
- [Rethinking "Batch" in BatchNorm](https://mlsanity.com/paper/2105.07576/) (2021)
- [Low-Rank Quantization-Aware Training for LLMs](https://mlsanity.com/paper/2406.06385/) (2024)
- [MQBench: Towards Reproducible and Deployable Model Quantization Benchmark](https://mlsanity.com/paper/2111.03759/) (2021)
- [Towards Efficient Post-training Quantization of Pre-trained Language Models](https://mlsanity.com/paper/2109.15082/) (2021)
- [Edge-Cloud Polarization and Collaboration: A Comprehensive Survey for AI](https://mlsanity.com/paper/2111.06061/) (2021)
- [CBQ: Cross-Block Quantization for Large Language Models](https://mlsanity.com/paper/2312.07950/) (2023)
- [Post-training Quantization for Neural Networks with Provable Guarantees](https://mlsanity.com/paper/2201.11113/) (2022)
- [F8Net: Fixed-Point 8-bit Only Multiplication for Network Quantization](https://mlsanity.com/paper/2202.05239/) (2022)
- [Gradient-Free Structured Pruning with Unlabeled Data](https://mlsanity.com/paper/2303.04185/) (2023)
- [Post-training deep neural network pruning via layer-wise calibration](https://mlsanity.com/paper/2104.15023/) (2021)
- [AlphaTuning: Quantization-Aware Parameter-Efficient Adaptation of Large-Scale Pre-Trained Language Models](https://mlsanity.com/paper/2210.03858/) (2022)
- [EDA-DM: Enhanced Distribution Alignment for Post-Training Quantization of Diffusion Models](https://mlsanity.com/paper/2401.04585/) (2024)
- [Spectra: Surprising Effectiveness of Pretraining Ternary Language Models at Scale](https://mlsanity.com/paper/2407.12327/) (2024)
- [MPQ-DM: Mixed Precision Quantization for Extremely Low Bit Diffusion Models](https://mlsanity.com/paper/2412.11549/) (2024)
- [Nearest is Not Dearest: Towards Practical Defense against Quantization-conditioned Backdoor Attacks](https://mlsanity.com/paper/2405.12725/) (2024)
- [Solving Oscillation Problem in Post-Training Quantization Through a Theoretical Perspective](https://mlsanity.com/paper/2303.11906/) (2023)
- [FPTQuant: Function-Preserving Transforms for LLM Quantization](https://mlsanity.com/paper/2506.04985/) (2025)

## Abstract (arXiv)

Lately, post-training quantization methods have gained considerable attention, as they are simple to use, and require only a small unlabeled calibration set. This small dataset cannot be used to fine-tune the model without significant over-fitting. Instead, these methods only use the calibration set to set the activations' dynamic ranges. However, such methods always resulted in significant accuracy degradation, when used below 8-bits (except on small datasets). Here we aim to break the 8-bit barrier. To this end, we minimize the quantization errors of each layer separately by optimizing its parameters over the calibration set. We empirically demonstrate that this approach is: (1) much less susceptible to over-fitting than the standard fine-tuning approaches, and can be used even on a very small calibration set; and (2) more powerful than previous methods, which only set the activations' dynamic ranges. Furthermore, we demonstrate how to optimally allocate the bit-widths for each layer, while constraining accuracy degradation or model compression by proposing a novel integer programming formulation. Finally, we suggest model global statistics tuning, to correct biases introduced during quantization. Together, these methods yield state-of-the-art results for both vision and text models. For instance, on ResNet50, we obtain less than 1% accuracy degradation --- with 4-bit weights and activations in all layers, but the smallest two. We open-sourced our code.