# Neural Network Quantization for Efficient Inference: A Survey

arXiv: https://arxiv.org/abs/2112.06126 · HTML: https://mlsanity.com/paper/2112.06126/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 45 references that are in the map.
- First posted on arXiv 8 Dec 2021.

## Builds on (most cited first, 28 of 28)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation](https://mlsanity.com/paper/1308.3432/) (2013)
- [Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference](https://mlsanity.com/paper/1712.05877/) (2017)
- [XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks](https://mlsanity.com/paper/1603.05279/) (2016)
- [A Survey of Quantization Methods for Efficient Neural Network Inference](https://mlsanity.com/paper/2103.13630/) (2021)
- [Up or Down? Adaptive Rounding for Post-Training Quantization](https://mlsanity.com/paper/2004.10568/) (2020)
- [BinaryConnect: Training Deep Neural Networks with binary weights during propagations](https://mlsanity.com/paper/1511.00363/) (2015)
- [DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients](https://mlsanity.com/paper/1606.06160/) (2016)
- [PACT: Parameterized Clipping Activation for Quantized Neural Networks](https://mlsanity.com/paper/1805.06085/) (2018)
- [HAQ: Hardware-Aware Automated Quantization with Mixed Precision](https://mlsanity.com/paper/1811.08886/) (2018)
- [HAWQ: Hessian AWare Quantization of Neural Networks with Mixed-Precision](https://mlsanity.com/paper/1905.03696/) (2019)
- [Deep Learning with Limited Numerical Precision](https://mlsanity.com/paper/1502.02551/) (2015)
- [HAWQ-V2: Hessian Aware trace-Weighted Quantization of Neural Networks](https://mlsanity.com/paper/1911.03852/) (2019)
- [Predicting Parameters in Deep Learning](https://mlsanity.com/paper/1306.0543/) (2013)
- [Trained Ternary Quantization](https://mlsanity.com/paper/1612.01064/) (2016)
- [Ternary Weight Networks](https://mlsanity.com/paper/1605.04711/) (2016)
- [Maxout Networks](https://mlsanity.com/paper/1302.4389/) (2013)
- [Training with Quantization Noise for Extreme Model Compression](https://mlsanity.com/paper/2004.07320/) (2020)
- [Fixed Point Quantization of Deep Convolutional Networks](https://mlsanity.com/paper/1511.06393/) (2015)
- [Apprentice: Using Knowledge Distillation Techniques To Improve Low-Precision Network Accuracy](https://mlsanity.com/paper/1711.05852/) (2017)
- [Towards Effective Low-bitwidth Convolutional Neural Networks](https://mlsanity.com/paper/1711.00205/) (2017)
- [HAWQV3: Dyadic Neural Network Quantization](https://mlsanity.com/paper/2011.10680/) (2020)
- [BSQ: Exploring Bit-Level Sparsity for Mixed-Precision Neural Network Quantization](https://mlsanity.com/paper/2102.10462/) (2021)
- [Ps and Qs: Quantization-aware pruning for efficient low latency neural network inference](https://mlsanity.com/paper/2102.11289/) (2021)
- [Mix and Match: A Novel FPGA-Centric Deep Neural Network Quantization Framework](https://mlsanity.com/paper/2012.04240/) (2020)
- [Highly Efficient 8-bit Low Precision Inference of Convolutional Neural Networks with IntelCaffe](https://mlsanity.com/paper/1805.08691/) (2018)

## Built on by (most cited first, 14 of 14)

- [Watt For What: Rethinking Deep Learning's Energy-Performance Relationship](https://mlsanity.com/paper/2310.06522/) (2023)
- [Reducing Computational Complexity of Neural Networks in Optical Channel Equalization: From Concepts to Implementation](https://mlsanity.com/paper/2208.12866/) (2022)
- [Efficient Deployment of Spiking Neural Networks on SpiNNaker2 for DVS Gesture Recognition Using Neuromorphic Intermediate Representation](https://mlsanity.com/paper/2504.06748/) (2025)
- [Low Precision Quantization-aware Training in Spiking Neural Networks with Differentiable Quantization Function](https://mlsanity.com/paper/2305.19295/) (2023)
- [Model Compression Methods for YOLOv5: A Review](https://mlsanity.com/paper/2307.11904/) (2023)
- [How Controlling the Variance can Improve Training Stability of Sparsely Activated DNNs and CNNs](https://mlsanity.com/paper/2602.05779/) (2026)
- [Efficient Split Learning LSTM Models for FPGA-based Edge IoT Devices](https://mlsanity.com/paper/2502.08692/) (2025)
- [Lyapunov-Guided Training for Hardware-Safe Neural Networks Under Fixed-Point Arithmetic](https://mlsanity.com/paper/2607.04531/) (2026)
- [Neural Architecture Codesign for Fast Bragg Peak Analysis](https://mlsanity.com/paper/2312.05978/) (2023)
- [Artificial Neural Networks for Photonic Applications: From Algorithms to Implementation](https://mlsanity.com/paper/2408.02685/) (2024)
- [Automatic mixed precision for optimizing gained time with constrained loss mean-squared-error based on model partition to sequential sub-graphs](https://mlsanity.com/paper/2505.13060/) (2025)
- [A Selective Quantization Tuner for ONNX Models](https://mlsanity.com/paper/2507.12196/) (2025)
- [Vanishing Contributions: A Unified Framework for Smooth and Iterative Model Compression](https://mlsanity.com/paper/2510.09696/) (2025)
- [SNAC-Pack 2.0: Scaled-Out Surrogate Neural Architecture Codesign](https://mlsanity.com/paper/2605.16138/) (2026)

## Abstract (arXiv)

As neural networks have become more powerful, there has been a rising desire to deploy them in the real world; however, the power and accuracy of neural networks is largely due to their depth and complexity, making them difficult to deploy, especially in resource-constrained devices. Neural network quantization has recently arisen to meet this demand of reducing the size and complexity of neural networks by reducing the precision of a network. With smaller and simpler networks, it becomes possible to run neural networks within the constraints of their target hardware. This paper surveys the many neural network quantization techniques that have been developed in the last decade. Based on this survey and comparison of neural network quantization techniques, we propose future directions of research in the area.