Automated Backend-Aware Post-Training Quantization
Not on the map. Why?
Abstract (arXiv)
Quantization is a key technique to reduce the resource requirement and improve the performance of neural network deployment. However, different hardware backends such as x86 CPU, NVIDIA GPU, ARM CPU, and accelerators may demand different implementations for quantized networks. This diversity calls for specialized post-training quantization pipelines to built for each hardware target, an engineering effort that is often too large for developers to keep up with. We tackle this problem with an automated post-training quantization framework called HAGO. HAGO provides a set of general quantization graph transformations based on a user-defined hardware specification and implements a search mechanism to find the optimal quantization strategy while satisfying hardware constraints for any model. We observe that HAGO achieves speedups of 2.09x, 1.97x, and 2.48x on Intel Xeon Cascade Lake CPUs, NVIDIA Tesla T4 GPUs, ARM Cortex-A CPUs on Raspberry Pi4 relative to full precision respectively, while maintaining the highest reported post-training quantization accuracy in each case.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 17 of its 25 references that are in the map. First posted on arXiv 27 Mar 2021.
Builds on
17 of 25 references.
References · 17
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
- 2015Distilling the Knowledge in a Neural Networkcited by 10,076
- 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
- 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
- 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
- 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
- 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,548
- 2019Low-bit Quantization of Neural Networks for Efficient Inferencecited by 185
- 2018Learning to Optimize Tensor Programscited by 138
- 2016Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1cited by 102
5 more
- 2018Glow: Graph Lowering Compiler Techniques for Neural Networkscited by 73
- 2020MLIR: A Compiler Infrastructure for the End of Moore's Lawcited by 64
- 2018TVM: An Automated End-to-End Optimizing Compiler for Deep Learningcited by 62
- 2019Same, Same But Different - Recovering Neural Network Quantization Error Through Weight Factorizationcited by 45
- 2020Efficient Execution of Quantized Deep Learning Models: A Compiler Approachcited by 12
Built on by
Citing papers per year: 2022: 1; 2023: 1.
2 papers.