# Learning to Quantize Deep Networks by Optimizing Quantization Intervals with Task Loss

arXiv: https://arxiv.org/abs/1808.05779 · HTML: https://mlsanity.com/paper/1808.05779/

## Key facts

- Cited by 160 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Aug 2018.

## Built on by (most cited first, 30 of 160)

- [A Survey of Quantization Methods for Efficient Neural Network Inference](https://mlsanity.com/paper/2103.13630/) (2021)
- [SqueezeLLM: Dense-and-Sparse Quantization](https://mlsanity.com/paper/2306.07629/) (2023)
- [Pruning and Quantization for Deep Neural Network Acceleration: A Survey](https://mlsanity.com/paper/2101.09671/) (2021)
- [RPTQ: Reorder-based Post-training Quantization for Large Language Models](https://mlsanity.com/paper/2304.01089/) (2023)
- [Q-ViT: Accurate and Fully Quantized Low-bit Vision Transformer](https://mlsanity.com/paper/2210.06707/) (2022)
- [Memory-Efficient Fine-Tuning of Compressed Large Language Models via sub-4-bit Integer Quantization](https://mlsanity.com/paper/2305.14152/) (2023)
- [OliVe: Accelerating Large Language Models via Hardware-friendly Outlier-Victim Pair Quantization](https://mlsanity.com/paper/2304.07493/) (2023)
- [Network Quantization with Element-wise Gradient Scaling](https://mlsanity.com/paper/2104.00903/) (2021)
- [BitDistiller: Unleashing the Potential of Sub-4-Bit LLMs via Self-Distillation](https://mlsanity.com/paper/2402.10631/) (2024)
- [Temporal Dynamic Quantization for Diffusion Models](https://mlsanity.com/paper/2306.02316/) (2023)
- [Nonuniform-to-Uniform Quantization: Towards Accurate Quantization via Generalized Straight-Through Estimation](https://mlsanity.com/paper/2111.14826/) (2021)
- [PD-Quant: Post-Training Quantization based on Prediction Difference Metric](https://mlsanity.com/paper/2212.07048/) (2022)
- [QLLM: Accurate and Efficient Low-Bitwidth Quantization for Large Language Models](https://mlsanity.com/paper/2310.08041/) (2023)
- [Mixed-Precision Neural Network Quantization via Learned Layer-wise Importance](https://mlsanity.com/paper/2203.08368/) (2022)
- [BiT: Robustly Binarized Multi-distilled Transformer](https://mlsanity.com/paper/2205.13016/) (2022)
- [ANT: Exploiting Adaptive Numerical Data Type for Low-bit Deep Neural Network Quantization](https://mlsanity.com/paper/2208.14286/) (2022)
- [FlexRound: Learnable Rounding based on Element-wise Division for Post-Training Quantization](https://mlsanity.com/paper/2306.00317/) (2023)
- [Differentiable Model Compression via Pseudo Quantization Noise](https://mlsanity.com/paper/2104.09987/) (2021)
- [Revisiting the Parameter Efficiency of Adapters from the Perspective of Precision Redundancy](https://mlsanity.com/paper/2307.16867/) (2023)
- [IntraQ: Learning Synthetic Images with Intra-Class Heterogeneity for Zero-Shot Network Quantization](https://mlsanity.com/paper/2111.09136/) (2021)
- [Learnable Companding Quantization for Accurate Low-bit Neural Networks](https://mlsanity.com/paper/2103.07156/) (2021)
- [MQBench: Towards Reproducible and Deployable Model Quantization Benchmark](https://mlsanity.com/paper/2111.03759/) (2021)
- [Hardware Approximate Techniques for Deep Neural Network Accelerators: A Survey](https://mlsanity.com/paper/2203.08737/) (2022)
- [Lightweight Deep Learning for Resource-Constrained Environments: A Survey](https://mlsanity.com/paper/2404.07236/) (2024)
- [Q-ViT: Fully Differentiable Quantization for Vision Transformer](https://mlsanity.com/paper/2201.07703/) (2022)
- [NIPQ: Noise proxy-based Integrated Pseudo-Quantization](https://mlsanity.com/paper/2206.00820/) (2022)
- [A Survey of Techniques for Optimizing Transformer Inference](https://mlsanity.com/paper/2307.07982/) (2023)
- [Dynamic Perceiver for Efficient Visual Recognition](https://mlsanity.com/paper/2306.11248/) (2023)
- [Distribution Adaptive INT8 Quantization for Training CNNs](https://mlsanity.com/paper/2102.04782/) (2021)
- [SDQ: Stochastic Differentiable Quantization with Mixed Precision](https://mlsanity.com/paper/2206.04459/) (2022)

## Abstract (arXiv)

Reducing bit-widths of activations and weights of deep networks makes it efficient to compute and store them in memory, which is crucial in their deployments to resource-limited devices, such as mobile phones. However, decreasing bit-widths with quantization generally yields drastically degraded accuracy. To tackle this problem, we propose to learn to quantize activations and weights via a trainable quantizer that transforms and discretizes them. Specifically, we parameterize the quantization intervals and obtain their optimal values by directly minimizing the task loss of the network. This quantization-interval-learning (QIL) allows the quantized networks to maintain the accuracy of the full-precision (32-bit) networks with bit-width as low as 4-bit and minimize the accuracy degeneration with further bit-width reduction (i.e., 3 and 2-bit). Moreover, our quantizer can be trained on a heterogeneous dataset, and thus can be used to quantize pretrained networks without access to their training data. We demonstrate the effectiveness of our trainable quantizer on ImageNet dataset with various network architectures such as ResNet-18, -34 and AlexNet, on which it outperforms existing methods to achieve the state-of-the-art accuracy.