# Discovering Low-Precision Networks Close to Full-Precision Networks for Efficient Embedded Inference

arXiv: https://arxiv.org/abs/1809.04191 · HTML: https://mlsanity.com/paper/1809.04191/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Sep 2018.

## Built on by (most cited first, 30 of 33)

- [A Survey of Quantization Methods for Efficient Neural Network Inference](https://mlsanity.com/paper/2103.13630/) (2021)
- [Outlier Suppression: Pushing the Limit of Low-bit Transformer Language Models](https://mlsanity.com/paper/2209.13325/) (2022)
- [VS-Quant: Per-vector Scaled Quantization for Accurate Low-Precision Neural Network Inference](https://mlsanity.com/paper/2102.04503/) (2021)
- [Zero-shot Adversarial Quantization](https://mlsanity.com/paper/2103.15263/) (2021)
- [MQBench: Towards Reproducible and Deployable Model Quantization Benchmark](https://mlsanity.com/paper/2111.03759/) (2021)
- [Pareto-Optimal Quantized ResNet Is Mostly 4-bit](https://mlsanity.com/paper/2105.03536/) (2021)
- [Differentiable Dynamic Quantization with Mixed Precision and Adaptive Resolution](https://mlsanity.com/paper/2106.02295/) (2021)
- [LUT-NN: Empower Efficient Neural Network Inference with Centroid Learning and Table Lookup](https://mlsanity.com/paper/2302.03213/) (2023)
- [Boost Vision Transformer with GPU-Friendly Sparsity and Quantization](https://mlsanity.com/paper/2305.10727/) (2023)
- [Dynamic Precision Analog Computing for Neural Networks](https://mlsanity.com/paper/2102.06365/) (2021)
- [QUARK: Quantization-Enabled Circuit Sharing for Transformer Acceleration by Exploiting Common Patterns in Nonlinear Operations](https://mlsanity.com/paper/2511.06767/) (2025)
- [Quantization-Guided Training for Compact TinyML Models](https://mlsanity.com/paper/2103.06231/) (2021)
- [Confounding Tradeoffs for Neural Network Quantization](https://mlsanity.com/paper/2102.06366/) (2021)
- [Mixed-Precision Neural Networks: A Survey](https://mlsanity.com/paper/2208.06064/) (2022)
- [MWQ: Multiscale Wavelet Quantized Neural Networks](https://mlsanity.com/paper/2103.05363/) (2021)
- [TENT: Efficient Quantization of Neural Networks on the tiny Edge with Tapered FixEd PoiNT](https://mlsanity.com/paper/2104.02233/) (2021)
- [Low-Precision Mixed-Computation Models for Inference on Edge](https://mlsanity.com/paper/2312.02210/) (2023)
- [Elastic-Link for Binarized Neural Network](https://mlsanity.com/paper/2112.10149/) (2021)
- [LG-LSQ: Learned Gradient Linear Symmetric Quantization](https://mlsanity.com/paper/2202.09009/) (2022)
- [QFT: Post-training quantization via fast joint finetuning of all degrees of freedom](https://mlsanity.com/paper/2212.02634/) (2022)
- [Robust Training of Neural Networks at Arbitrary Precision and Sparsity](https://mlsanity.com/paper/2409.09245/) (2024)
- [An Efficient Matrix Multiplication Algorithm for Accelerating Inference in Binary and Ternary Neural Networks](https://mlsanity.com/paper/2411.06360/) (2024)
- [Efficient Visual Recognition with Deep Neural Networks: A Survey on Recent Advances and New Directions](https://mlsanity.com/paper/2108.13055/) (2021)
- [Tabula: Efficiently Computing Nonlinear Activation Functions for Secure Neural Network Inference](https://mlsanity.com/paper/2203.02833/) (2022)
- [PalQuant: Accelerating High-precision Networks on Low-precision Accelerators](https://mlsanity.com/paper/2208.01944/) (2022)
- [Integer-Only Neural Network Quantization Scheme Based on Shift-Batch-Normalization](https://mlsanity.com/paper/2106.00127/) (2021)
- [Sensitivity-Aware Finetuning for Accuracy Recovery on Deep Learning Hardware](https://mlsanity.com/paper/2306.03076/) (2023)
- [Infrared Domain Adaptation with Zero-Shot Quantization](https://mlsanity.com/paper/2408.13925/) (2024)
- [Regularization-based Framework for Quantization-, Fault- and Variability-Aware Training](https://mlsanity.com/paper/2503.01297/) (2025)
- [Precision Neural Network Quantization via Learnable Adaptive Modules](https://mlsanity.com/paper/2504.17263/) (2025)

## Abstract (arXiv)

To realize the promise of ubiquitous embedded deep network inference, it is essential to seek limits of energy and area efficiency. To this end, low-precision networks offer tremendous promise because both energy and area scale down quadratically with the reduction in precision. Here we demonstrate ResNet-18, -34, -50, -152, Inception-v3, Densenet-161, and VGG-16bn networks on the ImageNet classification benchmark that, at 8-bit precision exceed the accuracy of the full-precision baseline networks after one epoch of finetuning, thereby leveraging the availability of pretrained models. We also demonstrate ResNet-18, -34, -50, -152, Densenet-161, and VGG-16bn 4-bit models that match the accuracy of the full-precision baseline networks -- the highest scores to date. Surprisingly, the weights of the low-precision networks are very close (in cosine similarity) to the weights of the corresponding baseline networks, making training from scratch unnecessary. We find that gradient noise due to quantization during training increases with reduced precision, and seek ways to overcome this noise. The number of iterations required by SGD to achieve a given training error is related to the square of (a) the distance of the initial solution from the final plus (b) the maximum variance of the gradient estimates. Therefore, we (a) reduce solution distance by starting with pretrained fp32 precision baseline networks and fine-tuning, and (b) combat gradient noise introduced by quantization by training longer and reducing learning rates. Sensitivity analysis indicates that these simple techniques, coupled with proper activation function range calibration to take full advantage of the limited precision, are sufficient to discover low-precision networks, if they exist, close to fp32 precision baseline networks. The results herein provide evidence that 4-bits suffice for classification.