mlsanityData to 14 Oct 2026

Scaled Quantization for the Vision Transformer

arXiv 2303.13601 ↗First posted 23 Mar 2023eess.IV cs.AR cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Quantization using a small number of bits shows promise for reducing latency and memory usage in deep neural networks. However, most quantization methods cannot readily handle complicated functions such as exponential and square root, and prior approaches involve complex training processes that must interact with floating-point values. This paper proposes a robust method for the full integer quantization of vision transformer networks without requiring any intermediate floating-point computations. The quantization techniques can be applied in various hardware or software implementations, including processor/memory architectures and FPGAs.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 15 of its 24 references that are in the map. First posted on arXiv 23 Mar 2023.

Builds on

15 of 24 references.

References · 15

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  4. 2020Training data-efficient image transformers & distillation through attentioncited by 4,586
  5. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,845
  6. 2016Gaussian Error Linear Units (GELUs)cited by 3,502
  7. 2016Xception: Deep Learning with Depthwise Separable Convolutionscited by 2,332
  8. 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,962
  9. 2019Q-BERT: Hessian Based Ultra Low Precision Quantization of BERTcited by 359
  10. 2019Q8BERT: Quantized 8Bit BERTcited by 300
  11. 2021Post-Training Quantization for Vision Transformercited by 278
  12. 2021Co-Scale Conv-Attentional Image Transformerscited by 224
3 more
  1. 2015Resiliency of Deep Neural Networks under Quantizationcited by 35
  2. 2020Towards Fully 8-bit Integer Inference for the Transformer Modelcited by 35
  3. 2020Extremely Low Bit Transformer Quantization for On-Device Neural Machine Translationcited by 14

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2021PTQ4ViT: Post-training quantization for vision transformers with twin uniform quantization
  2. 2024P2-ViT: Power-of-Two Post-Training Quantization and Acceleration for Fully Quantized Vision Transformer
  3. 2021A Survey of Quantization Methods for Efficient Neural Network Inference
  4. 2023Patch-wise Mixed-Precision Quantization of Vision Transformer
  5. 2025Low-bit Model Quantization for Deep Neural Networks: A Survey
  6. 2021FQ-ViT: Post-Training Quantization for Fully Quantized Vision Transformer
  7. 2024Model Quantization and Hardware Acceleration for Vision Transformers: A Comprehensive Survey
  8. 2023TSPTQ-ViT: Two-scaled post-training quantization for vision transformer