# Post-training 4-bit quantization of convolution networks for rapid-deployment

arXiv: https://arxiv.org/abs/1810.05723 · HTML: https://mlsanity.com/paper/1810.05723/

## Key facts

- Cited by 42 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Oct 2018.

## Built on by (most cited first, 30 of 42)

- [Outlier Suppression+: Accurate quantization of large language models by equivalent and optimal shifting and scaling](https://mlsanity.com/paper/2304.09145/) (2023)
- [Zero-shot Adversarial Quantization](https://mlsanity.com/paper/2103.15263/) (2021)
- [Auto-Split: A General Framework of Collaborative Edge-Cloud AI](https://mlsanity.com/paper/2108.13041/) (2021)
- [Learning from Students: Applying t-Distributions to Explore Accurate and Efficient Formats for LLMs](https://mlsanity.com/paper/2405.03103/) (2024)
- [Lightweight compression of neural network feature tensors for collaborative intelligence](https://mlsanity.com/paper/2105.06002/) (2021)
- [AutoReCon: Neural Architecture Search-based Reconstruction for Data-free Compression](https://mlsanity.com/paper/2105.12151/) (2021)
- [Wavelet Feature Maps Compression for Image-to-Image CNNs](https://mlsanity.com/paper/2205.12268/) (2022)
- [Generative Zero-shot Network Quantization](https://mlsanity.com/paper/2101.08430/) (2021)
- [Linear Log-Normal Attention with Unbiased Concentration](https://mlsanity.com/paper/2311.13541/) (2023)
- [Lightweight Compression of Intermediate Neural Network Features for Collaborative Intelligence](https://mlsanity.com/paper/2105.07102/) (2021)
- [Accelerating Deep Learning Model Inference on Arm CPUs with Ultra-Low Bit Quantization and Runtime](https://mlsanity.com/paper/2207.08820/) (2022)
- [EPTQ: Enhanced Post-Training Quantization via Hessian-guided Network-wise Optimization](https://mlsanity.com/paper/2309.11531/) (2023)
- [Causal-DFQ: Causality Guided Data-free Network Quantization](https://mlsanity.com/paper/2309.13682/) (2023)
- [Panacea: Novel DNN Accelerator using Accuracy-Preserving Asymmetric Quantization and Energy-Saving Bit-Slice Sparsity](https://mlsanity.com/paper/2412.10059/) (2024)
- [Sharpness-Aware Data Generation for Zero-shot Quantization](https://mlsanity.com/paper/2510.07018/) (2025)
- [Empirical Evaluation of Post-Training Quantization Methods for Language Tasks](https://mlsanity.com/paper/2210.16621/) (2022)
- [Robustness-Guided Image Synthesis for Data-Free Quantization](https://mlsanity.com/paper/2310.03661/) (2023)
- [FP=xINT:Representing Neural Networks via Low-Bit Series Basis Functions](https://mlsanity.com/paper/2412.06865/) (2024)
- [Compression for Better: A General and Stable Lossless Compression Framework](https://mlsanity.com/paper/2412.06868/) (2024)
- [Symmetry Regularization and Saturating Nonlinearity for Robust Quantization](https://mlsanity.com/paper/2208.00338/) (2022)
- [Efficient Adaptive Activation Rounding for Post-Training Quantization](https://mlsanity.com/paper/2208.11945/) (2022)
- [QFT: Post-training quantization via fast joint finetuning of all degrees of freedom](https://mlsanity.com/paper/2212.02634/) (2022)
- [Augmenting Hessians with Inter-Layer Dependencies for Mixed-Precision Post-Training Quantization](https://mlsanity.com/paper/2306.04879/) (2023)
- [Lossless Model Compression via Joint Low-Rank Factorization Optimization](https://mlsanity.com/paper/2412.06867/) (2024)
- [FxP-QNet: A Post-Training Quantizer for the Design of Mixed Low-Precision DNNs with Dynamic Fixed-Point Representation](https://mlsanity.com/paper/2203.12091/) (2022)
- [ACQ: Improving Generative Data-free Quantization Via Attention Correction](https://mlsanity.com/paper/2301.07266/) (2023)
- [Communication-Efficient Federated Learning via Clipped Uniform Quantization](https://mlsanity.com/paper/2405.13365/) (2024)
- [Art and Science of Quantizing Large-Scale Models: A Comprehensive Overview](https://mlsanity.com/paper/2409.11650/) (2024)
- [Arch-Net: Model Distillation for Architecture Agnostic Model Deployment](https://mlsanity.com/paper/2111.01135/) (2021)
- [BiVM: Accurate Binarized Neural Network for Efficient Video Matting](https://mlsanity.com/paper/2507.04456/) (2025)

## Abstract (arXiv)

Convolutional neural networks require significant memory bandwidth and storage for intermediate computations, apart from substantial computing resources. Neural network quantization has significant benefits in reducing the amount of intermediate results, but it often requires the full datasets and time-consuming fine tuning to recover the accuracy lost after quantization. This paper introduces the first practical 4-bit post training quantization approach: it does not involve training the quantized model (fine-tuning), nor it requires the availability of the full dataset. We target the quantization of both activations and weights and suggest three complementary methods for minimizing quantization error at the tensor level, two of whom obtain a closed-form analytical solution. Combining these methods, our approach achieves accuracy that is just a few percents less the state-of-the-art baseline across a wide range of convolutional models. The source code to replicate all experiments is available on GitHub: urlhttps://github.com/submission2019/cnn-quantization.