# A Comprehensive Survey on Model Quantization for Deep Neural Networks in Image Classification

arXiv: https://arxiv.org/abs/2205.07877 · HTML: https://mlsanity.com/paper/2205.07877/

## Key facts

- Cited by 65 papers in the map (data through 14 Oct 2026).
- Builds on 124 of its 195 references that are in the map.
- First posted on arXiv 14 May 2022.

## Builds on (most cited first, 30 of 124)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding](https://mlsanity.com/paper/1510.00149/) (2015)
- [Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation](https://mlsanity.com/paper/1308.3432/) (2013)
- [Learning both Weights and Connections for Efficient Neural Networks](https://mlsanity.com/paper/1506.02626/) (2015)
- [ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices](https://mlsanity.com/paper/1707.01083/) (2017)
- [Pruning Filters for Efficient ConvNets](https://mlsanity.com/paper/1608.08710/) (2016)
- [SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size](https://mlsanity.com/paper/1602.07360/) (2016)
- [XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks](https://mlsanity.com/paper/1603.05279/) (2016)
- [A Survey of Quantization Methods for Efficient Neural Network Inference](https://mlsanity.com/paper/2103.13630/) (2021)
- [Channel Pruning for Accelerating Very Deep Neural Networks](https://mlsanity.com/paper/1707.06168/) (2017)
- [Learned Step Size Quantization](https://mlsanity.com/paper/1902.08153/) (2019)
- [Up or Down? Adaptive Rounding for Post-Training Quantization](https://mlsanity.com/paper/2004.10568/) (2020)
- [Network In Network](https://mlsanity.com/paper/1312.4400/) (2013)
- [To prune, or not to prune: exploring the efficacy of pruning for model compression](https://mlsanity.com/paper/1710.01878/) (2017)
- [BinaryConnect: Training Deep Neural Networks with binary weights during propagations](https://mlsanity.com/paper/1511.00363/) (2015)
- [DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients](https://mlsanity.com/paper/1606.06160/) (2016)
- [BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstruction](https://mlsanity.com/paper/2102.05426/) (2021)
- [PACT: Parameterized Clipping Activation for Quantized Neural Networks](https://mlsanity.com/paper/1805.06085/) (2018)
- [HAQ: Hardware-Aware Automated Quantization with Mixed Precision](https://mlsanity.com/paper/1811.08886/) (2018)
- [ThiNet: A Filter Level Pruning Method for Deep Neural Network Compression](https://mlsanity.com/paper/1707.06342/) (2017)
- [Efficient Processing of Deep Neural Networks: A Tutorial and Survey](https://mlsanity.com/paper/1703.09039/) (2017)
- [Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations](https://mlsanity.com/paper/1609.07061/) (2016)

## Built on by (most cited first, 30 of 65)

- [Lightweight Deep Learning for Resource-Constrained Environments: A Survey](https://mlsanity.com/paper/2404.07236/) (2024)
- [Shifting AI Efficiency From Model-Centric to Data-Centric Compression](https://mlsanity.com/paper/2505.19147/) (2025)
- [FrameQuant: Flexible Low-Bit Quantization for Transformers](https://mlsanity.com/paper/2403.06082/) (2024)
- [Fourier Prompt Tuning for Modality-Incomplete Scene Segmentation](https://mlsanity.com/paper/2401.16923/) (2024)
- [Low-bit Model Quantization for Deep Neural Networks: A Survey](https://mlsanity.com/paper/2505.05530/) (2025)
- [Efficient Neural Networks for Tiny Machine Learning: A Comprehensive Review](https://mlsanity.com/paper/2311.11883/) (2023)
- [Quantization Meets Reasoning: Exploring and Mitigating Degradation of Low-Bit LLMs in Mathematical Reasoning](https://mlsanity.com/paper/2505.11574/) (2025)
- [Neural Network Quantization for Microcontrollers: A Comprehensive Survey of Methods, Platforms, and Applications](https://mlsanity.com/paper/2508.15008/) (2025)
- [Investigating Calibration and Corruption Robustness of Post-hoc Pruned Perception CNNs: An Image Classification Benchmark Study](https://mlsanity.com/paper/2405.20876/) (2024)
- [Dynamic layer selection in decoder-only transformers](https://mlsanity.com/paper/2410.20022/) (2024)
- [PATCH: Learnable Tile-level Hybrid Sparsity for LLMs](https://mlsanity.com/paper/2509.23410/) (2025)
- [MixBCT: Towards Self-Adapting Backward-Compatible Training](https://mlsanity.com/paper/2308.06948/) (2023)
- [ELIP: Efficient Discriminative Language-Image Pre-training with Fewer Vision Tokens](https://mlsanity.com/paper/2309.16738/) (2023)
- [Towards Green AI: Current status and future research](https://mlsanity.com/paper/2407.10237/) (2024)
- [Impact of ML Optimization Tactics on Greener Pre-Trained ML Models](https://mlsanity.com/paper/2409.12878/) (2024)
- [GCI-ViTAL: Gradual Confidence Improvement with Vision Transformers for Active Learning on Label Noise](https://mlsanity.com/paper/2411.05939/) (2024)
- [Test-time Cost-and-Quality Controllable Arbitrary-Scale Super-Resolution with Variable Fourier Components](https://mlsanity.com/paper/2412.05517/) (2024)
- [GLoT: A Novel Gated-Logarithmic Transformer for Efficient Sign Language Translation](https://mlsanity.com/paper/2502.12223/) (2025)
- [Agentic Satellite-Augmented Low-Altitude Economy and Terrestrial Networks: A Survey on Generative Approaches](https://mlsanity.com/paper/2507.14633/) (2025)
- [OPTIMA: Optimal One-shot Pruning for LLMs via Quadratic Programming Reconstruction](https://mlsanity.com/paper/2512.13886/) (2025)
- [RESQ: A Unified Framework for REliability- and Security Enhancement of Quantized Deep Neural Networks](https://mlsanity.com/paper/2603.15413/) (2026)
- [MixQuant: Adaptive Mixed-Precision Quantization for Large Language Models](https://mlsanity.com/paper/2607.23047/) (2026)
- [Predicting Probabilities of Error to Combine Quantization and Early Exiting: QuEE](https://mlsanity.com/paper/2406.14404/) (2024)
- [GreenMachine: Automatic Design of Zero-Cost Proxies for Energy-Efficient NAS](https://mlsanity.com/paper/2411.15290/) (2024)
- [Coded Deep Learning: Framework and Algorithm](https://mlsanity.com/paper/2501.09849/) (2025)
- [Hardware-Aware DNN Compression for Homogeneous Edge Devices](https://mlsanity.com/paper/2501.15240/) (2025)
- [Aggregating empirical evidence from data strategy studies: a case on model quantization](https://mlsanity.com/paper/2505.00816/) (2025)
- [Onboard Optimization and Learning: A Survey](https://mlsanity.com/paper/2505.08793/) (2025)
- [PhD Thesis Summary: Methods for Reliability Assessment and Enhancement of Deep Neural Network Hardware Accelerators](https://mlsanity.com/paper/2603.08724/) (2026)
- [FEMBA on the Edge: Physiologically-Aware Pre-Training, Quantization, and Deployment of a Bidirectional Mamba EEG Foundation Model on an Ultra-low Power Microcontroller](https://mlsanity.com/paper/2603.26716/) (2026)

## Abstract (arXiv)

Recent advancements in machine learning achieved by Deep Neural Networks (DNNs) have been significant. While demonstrating high accuracy, DNNs are associated with a huge number of parameters and computations, which leads to high memory usage and energy consumption. As a result, deploying DNNs on devices with constrained hardware resources poses significant challenges. To overcome this, various compression techniques have been widely employed to optimize DNN accelerators. A promising approach is quantization, in which the full-precision values are stored in low bit-width precision. Quantization not only reduces memory requirements but also replaces high-cost operations with low-cost ones. DNN quantization offers flexibility and efficiency in hardware design, making it a widely adopted technique in various methods. Since quantization has been extensively utilized in previous works, there is a need for an integrated report that provides an understanding, analysis, and comparison of different quantization approaches. Consequently, we present a comprehensive survey of quantization concepts and methods, with a focus on image classification. We describe clustering-based quantization methods and explore the use of a scale factor parameter for approximating full-precision values. Moreover, we thoroughly review the training of a quantized DNN, including the use of a straight-through estimator and quantization regularization. We explain the replacement of floating-point operations with low-cost bitwise operations in a quantized DNN and the sensitivity of different layers in quantization. Furthermore, we highlight the evaluation metrics for quantization methods and important benchmarks in the image classification task. We also present the accuracy of the state-of-the-art methods on CIFAR-10 and ImageNet.