# Value-aware Quantization for Training and Inference of Neural Networks

arXiv: https://arxiv.org/abs/1804.07802 · HTML: https://mlsanity.com/paper/1804.07802/

## Key facts

- Cited by 52 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Apr 2018.

## Built on by (most cited first, 30 of 52)

- [A Survey of Quantization Methods for Efficient Neural Network Inference](https://mlsanity.com/paper/2103.13630/) (2021)
- [I-BERT: Integer-only BERT Quantization](https://mlsanity.com/paper/2101.01321/) (2021)
- [BiLLM: Pushing the Limit of Post-Training Quantization for LLMs](https://mlsanity.com/paper/2402.04291/) (2024)
- [OWQ: Outlier-Aware Weight Quantization for Efficient Fine-Tuning and Inference of Large Language Models](https://mlsanity.com/paper/2306.02272/) (2023)
- [PD-Quant: Post-Training Quantization based on Prediction Difference Metric](https://mlsanity.com/paper/2212.07048/) (2022)
- [Diversifying Sample Generation for Accurate Data-Free Quantization](https://mlsanity.com/paper/2103.01049/) (2021)
- [BSQ: Exploring Bit-Level Sparsity for Mixed-Precision Neural Network Quantization](https://mlsanity.com/paper/2102.10462/) (2021)
- [OMPQ: Orthogonal Mixed Precision Quantization](https://mlsanity.com/paper/2109.07865/) (2021)
- [DKM: Differentiable K-Means Clustering Layer for Neural Network Compression](https://mlsanity.com/paper/2108.12659/) (2021)
- [F8Net: Fixed-Point 8-bit Only Multiplication for Network Quantization](https://mlsanity.com/paper/2202.05239/) (2022)
- [EMQ: Evolving Training-free Proxies for Automated Mixed Precision Quantization](https://mlsanity.com/paper/2307.10554/) (2023)
- [Distance-aware Quantization](https://mlsanity.com/paper/2108.06983/) (2021)
- [Post-Training Sparsity-Aware Quantization](https://mlsanity.com/paper/2105.11010/) (2021)
- [Digital-SC: Digital Semantic Communication with Adaptive Network Split and Learned Non-Linear Quantization](https://mlsanity.com/paper/2305.13553/) (2023)
- [SLiM: One-shot Quantization and Sparsity with Low-rank Approximation for LLM Weight Compression](https://mlsanity.com/paper/2410.09615/) (2024)
- [LANA: Latency Aware Network Acceleration](https://mlsanity.com/paper/2107.10624/) (2021)
- [Diverse Sample Generation: Pushing the Limit of Generative Data-free Quantization](https://mlsanity.com/paper/2109.00212/) (2021)
- [Low-Precision Training of Large Language Models: Methods, Challenges, and Opportunities](https://mlsanity.com/paper/2505.01043/) (2025)
- [HPTQ: Hardware-Friendly Post Training Quantization](https://mlsanity.com/paper/2109.09113/) (2021)
- [Q-LIC: Quantizing Learned Image Compression with Channel Splitting](https://mlsanity.com/paper/2205.14510/) (2022)
- [HGQ: High Granularity Quantization for Real-time Neural Networks on FPGAs](https://mlsanity.com/paper/2405.00645/) (2024)
- [ViT-1.58b: Mobile Vision Transformers in the 1-bit Era](https://mlsanity.com/paper/2406.18051/) (2024)
- [Lightweight Compression of Intermediate Neural Network Features for Collaborative Intelligence](https://mlsanity.com/paper/2105.07102/) (2021)
- [Mixed-Precision Neural Networks: A Survey](https://mlsanity.com/paper/2208.06064/) (2022)
- [Random and Adversarial Bit Error Robustness: Energy-Efficient and Secure DNN Accelerators](https://mlsanity.com/paper/2104.08323/) (2021)
- [RMSMP: A Novel Deep Neural Network Quantization Framework with Row-wise Mixed Schemes and Multiple Precisions](https://mlsanity.com/paper/2111.00153/) (2021)
- [A Generalized Zero-Shot Quantization of Deep Convolutional Neural Networks via Learned Weights Statistics](https://mlsanity.com/paper/2112.02834/) (2021)
- [Machine Learning Sensors](https://mlsanity.com/paper/2206.03266/) (2022)
- [Analysis of Quantization on MLP-based Vision Models](https://mlsanity.com/paper/2209.06383/) (2022)
- [Rethinking Floating Point Overheads for Mixed Precision DNN Accelerators](https://mlsanity.com/paper/2101.11748/) (2021)

## Abstract (arXiv)

We propose a novel value-aware quantization which applies aggressively reduced precision to the majority of data while separately handling a small amount of large data in high precision, which reduces total quantization errors under very low precision. We present new techniques to apply the proposed quantization to training and inference. The experiments show that our method with 3-bit activations (with 2% of large ones) can give the same training accuracy as full-precision one while offering significant (41.6% and 53.7%) reductions in the memory cost of activations in ResNet-152 and Inception-v3 compared with the state-of-the-art method. Our experiments also show that deep networks such as Inception-v3, ResNet-101 and DenseNet-121 can be quantized for inference with 4-bit weights and activations (with 1% 16-bit data) within 1% top-1 accuracy drop.