mlsanityData to 7 Oct 2026

P2-ViT: Power-of-Two Post-Training Quantization and Acceleration for Fully Quantized Vision Transformer

arXiv 2405.19915 ↗First posted 30 May 2024cs.AIcited by 4

Not on the map. Why?

Abstract (arXiv)

Vision Transformers (ViTs) have excelled in computer vision tasks but are memory-consuming and computation-intensive, challenging their deployment on resource-constrained devices. To tackle this limitation, prior works have explored ViT-tailored quantization algorithms but retained floating-point scaling factors, which yield non-negligible re-quantization overhead, limiting ViTs' hardware efficiency and motivating more hardware-friendly solutions. To this end, we propose P2-ViT, the first underlinePower-of-Two (PoT) underlinepost-training quantization and acceleration framework to accelerate fully quantized ViTs. Specifically, as for quantization, we explore a dedicated quantization scheme to effectively quantize ViTs with PoT scaling factors, thus minimizing the re-quantization overhead. Furthermore, we propose coarse-to-fine automatic mixed-precision quantization to enable better accuracy-efficiency trade-offs. In terms of hardware, we develop a dedicated chunk-based accelerator featuring multiple tailored sub-processors to individually handle ViTs' different types of operations, alleviating reconfigurable overhead. Additionally, we design a tailored row-stationary dataflow to seize the pipeline processing opportunity introduced by our PoT scaling factors, thereby enhancing throughput. Extensive experiments consistently validate P2-ViT's effectiveness. Particularly, we offer comparable or even superior quantization performance with PoT scaling factors when compared to the counterpart with floating-point scaling factors. Besides, we achieve up to 10.1× speedup and 36.8× energy saving over GPU's Turing Tensor Cores, and up to 1.84× higher computation utilization efficiency against SOTA quantization-based ViT accelerators. Codes are available at urlhttps://github.com/shihuihong214/P2-ViT.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 37 of its 60 references that are in the map. First posted on arXiv 30 May 2024.

Builds on

37 of 60 references.

References · 37

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  4. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,548
  5. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,338
  6. 2020Up or Down? Adaptive Rounding for Post-Training Quantizationcited by 600
  7. 2018FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Searchcited by 480
  8. 2021LeViT: a Vision Transformer in ConvNet's Clothing for Faster Inferencecited by 475
  9. 2021BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstructioncited by 464
  10. 2019Q-BERT: Hessian Based Ultra Low Precision Quantization of BERTcited by 355
  11. 2021Post-Training Quantization for Vision Transformercited by 274
  12. 2019HAWQ-V2: Hessian Aware trace-Weighted Quantization of Neural Networkscited by 250
25 more
  1. 2021I-BERT: Integer-only BERT Quantizationcited by 250
  2. 2019Low-bit Quantization of Neural Networks for Efficient Inferencecited by 185
  3. 2021PTQ4ViT: Post-training quantization for vision transformers with twin uniform quantizationcited by 173
  4. 2023FLatten Transformer: Vision Transformer using Focused Linear Attentioncited by 170
  5. 2021FQ-ViT: Post-Training Quantization for Fully Quantized Vision Transformercited by 148
  6. 2022RepQ-ViT: Scale Reparameterization for Post-Training Quantization of Vision Transformerscited by 134
  7. 2022I-ViT: Integer-only Quantization for Efficient Vision Transformer Inferencecited by 93
  8. 2018Hessian-based Analysis of Large Batch Training and Robustness to Adversariescited by 87
  9. 2019Trained Quantization Thresholds for Accurate and Efficient Fixed-Point Inference of Deep Neural Networkscited by 87
  10. 2020HAWQV3: Dyadic Neural Network Quantizationcited by 59
  11. 2022NoisyQuant: Noisy Bias-Enhanced Post-Training Activation Quantization for Vision Transformerscited by 59
  12. 2020EasyQuant: Post-training Quantization via Scale Optimizationcited by 57
  13. 2019DeepShift: Towards Multiplication-Less Neural Networkscited by 52
  14. 2022HeatViT: Hardware-Efficient Adaptive Token Pruning for Vision Transformerscited by 46
  15. 2020Hardware Accelerator for Multi-Head Attention and Position-Wise Feed-Forward in the Transformercited by 42
  16. 2022ViTCoD: Vision Transformer Acceleration via Dedicated Algorithm and Accelerator Co-Designcited by 41
  17. 2021AlphaNet: Improved Training of Supernets with Alpha-Divergencecited by 36
  18. 2022ViTALiTy: Unifying Low-rank and Sparse Approximation for Vision Transformer Acceleration with a Linear Taylor Attentioncited by 29
  19. 2022Auto-ViT-Acc: An FPGA-Aware Automatic Acceleration Framework for Vision Transformer with Mixed-Scheme Quantizationcited by 28
  20. 2022VAQF: Fully Automatic Software-Hardware Co-Design Framework for Low-Bit Vision Transformercited by 27
  21. 2021Hardware Acceleration of Fully Quantized BERT for Efficient Natural Language Processingcited by 21
  22. 2018Large batch size training of neural networks with adversarial training and second-order informationcited by 18
  23. 2020DNN-Chip Predictor: An Analytical Performance Predictor for DNN Accelerators with Various Dataflows and Hardware Architecturescited by 16
  24. 2022RAPQ: Rescuing Accuracy for Power-of-Two Low-bit Post-training Quantizationcited by 14
  25. 2022NASA: Neural Architecture Search and Acceleration for Hardware Inspired Hybrid Networkscited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 1.

4 papers.

  1. 2025FQ-PETR: Fully Quantized Position Embedding Transformation for Multi-View 3D Object Detectioncited by 2
  2. 2025CoQMoE: Co-Designed Quantization and Computation Orchestration for Mixture-of-Experts Vision Transformer on FPGAcited by 0
  3. 2025FQ-PETR: Fully Quantized Position Embedding Transformation for Multi-View 3D Object Detectioncited by 0
  4. 2026Amortized-Precision Quantization for Early-Exit Vision Transformerscited by 0