mlsanityData to 7 Oct 2026

I-ViT: Integer-only Quantization for Efficient Vision Transformer Inference

arXiv 2207.01405 ↗First posted 4 Jul 2022cs.CVcited by 93

Not on the map. Why?

Abstract (arXiv)

Vision Transformers (ViTs) have achieved state-of-the-art performance on various computer vision applications. However, these models have considerable storage and computational overheads, making their deployment and efficient inference on edge devices challenging. Quantization is a promising approach to reducing model complexity, and the dyadic arithmetic pipeline can allow the quantized models to perform efficient integer-only inference. Unfortunately, dyadic arithmetic is based on the homogeneity condition in convolutional neural networks, which is not applicable to the non-linear components in ViTs, making integer-only inference of ViTs an open issue. In this paper, we propose I-ViT, an integer-only quantization scheme for ViTs, to enable ViTs to perform the entire computational graph of inference with integer arithmetic and bit-shifting, and without any floating-point arithmetic. In I-ViT, linear operations (e.g., MatMul and Dense) follow the integer-only pipeline with dyadic arithmetic, and non-linear operations (e.g., Softmax, GELU, and LayerNorm) are approximated by the proposed light-weight integer-only arithmetic methods. More specifically, I-ViT applies the proposed Shiftmax and ShiftGELU, which are designed to use integer bit-shifting to approximate the corresponding floating-point operations. We evaluate I-ViT on various benchmark models and the results show that integer-only INT8 quantization achieves comparable (or even slightly higher) accuracy to the full-precision (FP) baseline. Furthermore, we utilize TVM for practical hardware deployment on the GPU's integer arithmetic units, achieving 3.72~4.11× inference speedup compared to the FP model. Code of both Pytorch and TVM is released at https://github.com/zkkli/I-ViT.

Key facts. Cited by 93 papers in the map (data through 7 Oct 2026). Builds on 37 of its 54 references that are in the map. First posted on arXiv 4 Jul 2022.

Builds on

37 of 54 references.

References · 37

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  2. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  3. 2020End-to-End Object Detection with Transformerscited by 7,384
  4. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  5. 2016Gaussian Error Linear Units (GELUs)cited by 3,431
  6. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,410
  7. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,031
  8. 2017Decoupled Weight Decay Regularizationcited by 1,722
  9. 2021ViViT: A Video Vision Transformercited by 1,610
  10. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,548
  11. 2020Pre-Trained Image Processing Transformercited by 891
  12. 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 798
25 more
  1. 2020A Survey on Visual Transformercited by 653
  2. 2021Transformer in Transformercited by 632
  3. 2019Learned Step Size Quantizationcited by 604
  4. 2016DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradientscited by 557
  5. 2018Quantizing deep convolutional networks for efficient inference: A whitepapercited by 507
  6. 2018PACT: Parameterized Clipping Activation for Quantized Neural Networkscited by 456
  7. 2018HAQ: Hardware-Aware Automated Quantization with Mixed Precisioncited by 426
  8. 2019HAWQ: Hessian AWare Quantization of Neural Networks with Mixed-Precisioncited by 387
  9. 2021Post-Training Quantization for Vision Transformercited by 274
  10. 2018LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networkscited by 272
  11. 2021I-BERT: Integer-only BERT Quantizationcited by 250
  12. 2017Incremental Network Quantization: Towards Lossless CNNs with Low-Precision Weightscited by 211
  13. 2021Rethinking and Improving Relative Position Encoding for Vision Transformercited by 206
  14. 2020Binary Neural Networks: A Surveycited by 184
  15. 2020Integer Quantization for Deep Learning Inference: Principles and Empirical Evaluationcited by 183
  16. 2020LSQ+: Improving low-bit quantization through learnable offsets and better initializationcited by 180
  17. 2021FQ-ViT: Post-Training Quantization for Fully Quantized Vision Transformercited by 148
  18. 2019Additive Powers-of-Two Quantization: An Efficient Non-uniform Discretization for Neural Networkscited by 143
  19. 2021Patch Slimming for Efficient Vision Transformerscited by 131
  20. 2022MiniViT: Compressing Vision Transformers with Weight Multiplexingcited by 88
  21. 2021Learning Efficient Vision Transformers via Fine-Grained Manifold Distillationcited by 60
  22. 2022Patch Similarity Aware Data-Free Quantization for Vision Transformerscited by 55
  23. 2021Softermax: Hardware/Software Co-Design of an Efficient Softmax for Transformerscited by 52
  24. 2022Q-ViT: Fully Differentiable Quantization for Vision Transformercited by 36
  25. 2020Towards Fully 8-bit Integer Inference for the Transformer Modelcited by 35

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 4; 2023: 10; 2024: 36; 2025: 22; 2026: 21.

93 papers.

Sort:

By area:

  1. 2022RepQ-ViT: Scale Reparameterization for Post-Training Quantization of Vision Transformerscited by 134
  2. 2023Full Stack Optimization of Transformer Inference: a Surveycited by 79
  3. 2023Oscillation-free Quantization for Low-bit Vision Transformerscited by 55
  4. 2024A Survey on Transformer Compressioncited by 44
  5. 2022PSAQ-ViT V2: Towards Accurate and General Data-Free Quantization for Vision Transformerscited by 42
  6. 2023A Survey of Techniques for Optimizing Transformer Inferencecited by 35
  7. 2022BiViT: Extremely Compressed Binary Vision Transformercited by 28
  8. 2024Quantization without Tearscited by 24
  9. 2024EDA-DM: Enhanced Distribution Alignment for Post-Training Quantization of Diffusion Modelscited by 23
  10. 2024Towards Accurate Post-Training Quantization of Vision Transformers via Error Reductioncited by 23
  11. 2023I&S-ViT: An Inclusive & Stable Method for Pushing the Limit of Post-Training ViTs Quantizationcited by 19
  12. 2024I-LLM: Efficient Integer-Only Inference for Fully-Quantized Low-Bit Large Language Modelscited by 19
38 more
  1. 2024DopQ-ViT: Towards Distribution-Friendly and Outlier-Aware Post-Training Quantization for Vision Transformerscited by 19
  2. 2023Jumping through Local Minima: Quantization in the Loss Landscape of Vision Transformerscited by 18
  3. 2023SwiftTron: An Efficient Hardware Accelerator for Quantized Transformerscited by 17
  4. 2023Quantization Variation: A New Perspective on Training Transformers with Low-Bit Precisioncited by 17
  5. 2024CLAMP-ViT: Contrastive Data-Free Learning for Adaptive Post-Training Quantization of ViTscited by 17
  6. 2024A Comprehensive Survey of Convolutions in Deep Learning: Applications, Challenges, and Future Trendscited by 16
  7. 2024K-Sort Arena: Efficient and Reliable Benchmarking for Generative Models via K-wise Human Preferencescited by 16
  8. 2025FIMA-Q: Post-Training Quantization for Vision Transformers by Fisher Information Matrix Approximationcited by 16
  9. 2024Model Quantization and Hardware Acceleration for Vision Transformers: A Comprehensive Surveycited by 13
  10. 2025CacheQuant: Comprehensively Accelerated Diffusion Modelscited by 13
  11. 2024VLTP: Vision-Language Guided Token Pruning for Task-Oriented Segmentationcited by 11
  12. 2025DGQ: Distribution-Aware Group Quantization for Text-to-Image Diffusion Modelscited by 11
  13. 2025Low-bit Model Quantization for Deep Neural Networks: A Surveycited by 11
  14. 2023QFT: Quantized Full-parameter Tuning of LLMs with Affordable Resourcescited by 10
  15. 2024RepQuant: Towards Accurate Post-Training Quantization of Large Transformer Models via Scale Reparameterizationcited by 10
  16. 2025QUARK: Quantization-Enabled Circuit Sharing for Transformer Acceleration by Exploiting Common Patterns in Nonlinear Operationscited by 10
  17. 2024LRP-QViT: Mixed-Precision Vision Transformer Quantization via Layer-wise Relevance Propagationcited by 8
  18. 2024BitNet b1.58 Reloaded: State-of-the-art Performance Also on Smaller Networkscited by 8
  19. 2023A Model for Every User and Budget: Label-Free and Personalized Mixed-Precision Quantizationcited by 7
  20. 2024ViT-1.58b: Mobile Vision Transformers in the 1-bit Eracited by 7
  21. 2024MimiQ: Low-Bit Data-Free Quantization of Vision Transformers with Encouraging Inter-Head Attention Similaritycited by 7
  22. 2024Integer-only Quantized Transformers for Embedded FPGA-based Time-series Forecasting in AIoTcited by 6
  23. 2024Sorbet: A Neuromorphic Hardware-Compatible Transformer-Based Spiking Language Modelcited by 6
  24. 2024P2-ViT: Power-of-Two Post-Training Quantization and Acceleration for Fully Quantized Vision Transformercited by 4
  25. 2024Privacy-Preserving SAM Quantization for Efficient Edge Intelligence in Healthcarecited by 4
  26. 2024Slicing Vision Transformer for Flexible Inferencecited by 4
  27. 2025Continual Quantization-Aware Pre-Training: When to transition from 16-bit to 1.58-bit pre-training for BitNet language models?cited by 4
  28. 2024Trio-ViT: Post-Training Quantization and Acceleration for Softmax-Free Efficient Vision Transformercited by 3
  29. 2024Mixed Non-linear Quantization for Vision Transformerscited by 3
  30. 2024On Expressive Power of Quantized Neural Networks under Fixed-Point Arithmeticcited by 3
  31. 2024M2-ViT: Accelerating Hybrid Vision Transformers with Two-Level Mixed Quantizationcited by 3
  32. 2024Semantic Alignment and Reinforcement for Data-Free Quantization of Vision Transformerscited by 3
  33. 2025AHCQ-SAM: Toward Accurate and Hardware-Compatible Post-Training Segment Anything Model Quantizationcited by 3
  34. 2025SAQ-SAM: Semantically-Aligned Quantization for Segment Anything Modelcited by 3
  35. 2025LL-ViT: Edge Deployable Vision Transformers with Look Up Table Neuronscited by 3
  36. 2025IntAttention: A Fully Integer Attention Pipeline for Efficient Edge Inferencecited by 3
  37. 2026PTQ4ARVG: Post-Training Quantization for AutoRegressive Visual Generation Modelscited by 3
  38. 2026K-Sort Eval: Efficient Preference Evaluation for Visual Generation via Corrected VLM-as-a-Judgecited by 3

50 most cited of 93 All 93 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Token Compression Meets Compact Vision Transformers: A Survey and Comparative Evaluation for Edge AI
  2. 2024Quasar-ViT: Hardware-Oriented Quantization-Aware Architecture Search for Vision Transformers
  3. 2026ViT-5: Vision Transformers for The Mid-2020s
  4. 2021AdaViT: Adaptive Tokens for Efficient Vision Transformer
  5. 2025ViT-Linearizer: Distilling Quadratic Knowledge into Linear-Time Vision Models
  6. 2026Amortized-Precision Quantization for Early-Exit Vision Transformers
  7. 2023ShiftAddViT: Mixture of Multiplication Primitives Towards Efficient Vision Transformer
  8. 2025EdgeFlex-Transformer: Transformer Inference for Edge Devices