mlsanityData to 7 Oct 2026

PTQ4ViT: Post-training quantization for vision transformers with twin uniform quantization

arXiv 2111.12293 ↗First posted 24 Nov 2021cs.CVcited by 173

Not on the map. Why?

Abstract (arXiv)

Quantization is one of the most effective methods to compress neural networks, which has achieved great success on convolutional neural networks (CNNs). Recently, vision transformers have demonstrated great potential in computer vision. However, previous post-training quantization methods performed not well on vision transformer, resulting in more than 1% accuracy drop even in 8-bit quantization. Therefore, we analyze the problems of quantization on vision transformers. We observe the distributions of activation values after softmax and GELU functions are quite different from the Gaussian distribution. We also observe that common quantization metrics, such as MSE and cosine distance, are inaccurate to determine the optimal scaling factor. In this paper, we propose the twin uniform quantization method to reduce the quantization error on these activation values. And we propose to use a Hessian guided metric to evaluate different scaling factors, which improves the accuracy of calibration at a small cost. To enable the fast quantization of vision transformers, we develop an efficient framework, PTQ4ViT. Experiments show the quantized vision transformers achieve near-lossless prediction accuracy (less than 0.5% drop at 8-bit quantization) on the ImageNet classification task.

Key facts. Cited by 173 papers in the map (data through 7 Oct 2026). Builds on 23 of its 28 references that are in the map. First posted on arXiv 24 Nov 2021.

Builds on

23 of 28 references.

References · 23

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  6. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  7. 2020End-to-End Object Detection with Transformerscited by 7,384
  8. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  9. 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,960
  10. 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 798
  11. 2021BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstructioncited by 464
  12. 2018PACT: Parameterized Clipping Activation for Quantized Neural Networkscited by 456
11 more
  1. 2019Q-BERT: Hessian Based Ultra Low Precision Quantization of BERTcited by 355
  2. 2020MaX-DeepLab: End-to-End Panoptic Segmentation with Mask Transformerscited by 355
  3. 2019Data-Free Quantization Through Weight Equalization and Bias Correctioncited by 328
  4. 2021Post-Training Quantization for Vision Transformercited by 274
  5. 2018LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networkscited by 272
  6. 2019Low-bit Quantization of Neural Networks for Efficient Inferencecited by 185
  7. 2021Patch Slimming for Efficient Vision Transformerscited by 131
  8. 2020Post-Training Piecewise Linear Quantization for Deep Neural Networkscited by 77
  9. 2020EasyQuant: Post-training Quantization via Scale Optimizationcited by 57
  10. 2019Fully Quantized Transformer for Machine Translationcited by 36
  11. 2017Network Sketching: Exploiting Binary Structure in Deep CNNscited by 22

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 7; 2023: 23; 2024: 46; 2025: 49; 2026: 48.

173 papers.

Sort:

By area:

  1. 2022Post-training Quantization on Diffusion Modelscited by 262
  2. 2022RepQ-ViT: Scale Reparameterization for Post-Training Quantization of Vision Transformerscited by 134
  3. 2023RPTQ: Reorder-based Post-training Quantization for Large Language Modelscited by 112
  4. 2024ViDiT-Q: Efficient and Accurate Quantization of Diffusion Transformers for Image and Video Generationcited by 110
  5. 2024Efficient Multimodal Large Language Models: A Surveycited by 95
  6. 2024Q-DiT: Accurate Post-Training Quantization for Diffusion Transformerscited by 91
  7. 2024PTQ4DiT: Post-training Quantization for Diffusion Transformerscited by 77
  8. 2024Test-Time Model Adaptation with Only Forward Passescited by 74
  9. 2022PD-Quant: Post-Training Quantization based on Prediction Difference Metriccited by 73
  10. 2023Towards Accurate Post-Training Quantization for Vision Transformercited by 70
  11. 2024WKVQuant: Quantizing Weight and Key/Value Cache for Large Language Models Gains Morecited by 69
  12. 2023LLM-FP4: 4-Bit Floating-Point Quantized Transformerscited by 65
38 more
  1. 2022NoisyQuant: Noisy Bias-Enhanced Post-Training Activation Quantization for Vision Transformerscited by 59
  2. 2022Patch Similarity Aware Data-Free Quantization for Vision Transformerscited by 55
  3. 2023Revisiting the Parameter Efficiency of Adapters from the Perspective of Precision Redundancycited by 44
  4. 2024A Survey on Transformer Compressioncited by 44
  5. 2022PSAQ-ViT V2: Towards Accurate and General Data-Free Quantization for Vision Transformerscited by 42
  6. 2023A survey on efficient vision transformers: algorithms, techniques, and performance benchmarkingcited by 41
  7. 2023A Survey of Techniques for Optimizing Transformer Inferencecited by 35
  8. 2023Dynamic Perceiver for Efficient Visual Recognitioncited by 33
  9. 2023TinySAM: Pushing the Envelope for Efficient Segment Anything Modelcited by 33
  10. 2024PTQ4SAM: Post-Training Quantization for Segment Anythingcited by 32
  11. 2023LGViT: Dynamic Early Exiting for Accelerating Vision Transformercited by 30
  12. 2024A Survey of Low-bit Large Language Models: Basics, Systems, and Algorithmscited by 30
  13. 2024AdaLog: Post-Training Quantization for Vision Transformers with Adaptive Logarithm Quantizercited by 28
  14. 2023Latency-aware Unified Dynamic Networks for Efficient Image Recognitioncited by 26
  15. 2024Quantization without Tearscited by 24
  16. 2025Vision Transformers on the Edge: A Comprehensive Survey of Model Compression and Acceleration Strategiescited by 24
  17. 2024Towards Accurate Post-Training Quantization of Vision Transformers via Error Reductioncited by 23
  18. 2023I&S-ViT: An Inclusive & Stable Method for Pushing the Limit of Post-Training ViTs Quantizationcited by 19
  19. 2024DopQ-ViT: Towards Distribution-Friendly and Outlier-Aware Post-Training Quantization for Vision Transformerscited by 19
  20. 2023Jumping through Local Minima: Quantization in the Loss Landscape of Vision Transformerscited by 18
  21. 2024FrameQuant: Flexible Low-Bit Quantization for Transformerscited by 17
  22. 2024CLAMP-ViT: Contrastive Data-Free Learning for Adaptive Post-Training Quantization of ViTscited by 17
  23. 2025APHQ-ViT: Post-Training Quantization with Average Perturbation Hessian Based Reconstruction for Vision Transformerscited by 17
  24. 2025PMQ-VE: Progressive Multi-Frame Quantization for Video Enhancementcited by 17
  25. 2024On Efficient Variants of Segment Anything Model: A Surveycited by 16
  26. 2024LiteVAR: Compressing Visual Autoregressive Modelling with Efficient Attention and Quantizationcited by 16
  27. 2025FIMA-Q: Post-Training Quantization for Vision Transformers by Fisher Information Matrix Approximationcited by 16
  28. 2025Quantized Visual Geometry Grounded Transformercited by 16
  29. 2024Model Quantization and Hardware Acceleration for Vision Transformers: A Comprehensive Surveycited by 13
  30. 2024COMQ: A Backpropagation-Free Algorithm for Post-Training Quantizationcited by 12
  31. 2025Low-bit Model Quantization for Deep Neural Networks: A Surveycited by 11
  32. 2025QuantV2X: A Fully Quantized Multi-Agent System for Cooperative Perceptioncited by 11
  33. 2024MPTQ-ViT: Mixed-Precision Post-Training Quantization for Vision Transformercited by 10
  34. 2024RepQuant: Towards Accurate Post-Training Quantization of Large Transformer Models via Scale Reparameterizationcited by 10
  35. 2025LRQ-DiT: Log-Rotation Post-Training Quantization of Diffusion Transformers for Image and Video Generationcited by 10
  36. 2025QUARK: Quantization-Enabled Circuit Sharing for Transformer Acceleration by Exploiting Common Patterns in Nonlinear Operationscited by 10
  37. 2025Mix-QViT: Mixed-Precision Vision Transformer Quantization Driven by Layer Importance and Quantization Sensitivitycited by 9
  38. 2023Benchmarking the Reliability of Post-training Quantization: a Particular Focus on Worst-case Performancecited by 8

50 most cited of 173 All 173 (JSON)