Designing strong baselines for ternary neural network quantization through support and mass equalization
Not on the map. Why?
Abstract (arXiv)
Deep neural networks (DNNs) offer the highest performance in a wide range of applications in computer vision. These results rely on over-parameterized backbones, which are expensive to run. This computational burden can be dramatically reduced by quantizing (in either data-free (DFQ), post-training (PTQ) or quantization-aware training (QAT) scenarios) floating point values to ternary values (2 bits, with each weight taking value in {-1,0,1}). In this context, we observe that rounding to nearest minimizes the expected error given a uniform distribution and thus does not account for the skewness and kurtosis of the weight distribution, which strongly affects ternary quantization performance. This raises the following question: shall one minimize the highest or average quantization error? To answer this, we design two operators: TQuant and MQuant that correspond to these respective minimization tasks. We show experimentally that our approach allows to significantly improve the performance of ternary quantization through a variety of scenarios in DFQ, PTQ and QAT and give strong insights to pave the way for future research in deep neural network quantization.
Key facts. No citing papers in the map yet (data through 13 Oct 2026). Builds on 22 of its 28 references that are in the map. First posted on arXiv 30 Jun 2023.
Builds on
22 of 28 references.
References · 22
- 2015Deep Residual Learning for Image Recognitioncited by 51,950
- 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,426
- 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,786
- 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,435
- 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,585
- 2020Up or Down? Adaptive Rounding for Post-Training Quantizationcited by 617
- 2016DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradientscited by 565
- 2021BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstructioncited by 472
- 2019Data-Free Quantization Through Weight Equalization and Bias Correctioncited by 333
- 2019Understanding Straight-Through Estimator in Training Activation Quantized Neural Netscited by 228
- 2016Trained Ternary Quantizationcited by 208
- 2016Ternary Weight Networkscited by 204
10 more
- 2017Soft Weight-Sharing for Neural Network Compressioncited by 99
- 2018Training and Inference with Integers in Deep Neural Networkscited by 85
- 2022SQuant: On-the-Fly Data-Free Quantization via Diagonal Hessian Approximationcited by 55
- 2018Value-aware Quantization for Training and Inference of Neural Networkscited by 52
- 2019Same, Same But Different - Recovering Neural Network Quantization Error Through Weight Factorizationcited by 45
- 2018A Quantization-Friendly Separable Convolution for MobileNetscited by 22
- 2023PowerQuant: Automorphism Search for Non-Uniform Quantizationcited by 20
- 2021RED++ : Data-Free Pruning of Deep Neural Networks via Input Splitting and Output Mergingcited by 19
- 2022SPIQ: Data-Free Per-Channel Static Input Quantizationcited by 12
- 2022REx: Data-Free Residual Quantization Error Expansioncited by 6
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2023Ternary Quantization: A Survey
- 2021Pruning Ternary Quantization
- 2021A White Paper on Neural Network Quantization
- 2022Soft Threshold Ternary Networks
- 2025Low-bit Model Quantization for Deep Neural Networks: A Survey
- 2021A Survey of Quantization Methods for Efficient Neural Network Inference
- 2025Dual Precision Quantization for Efficient and Accurate Deep Neural Networks Inference
- 2023FLIQS: One-Shot Mixed-Precision Floating-Point and Integer Quantization Search