mlsanityData to 14 Oct 2026

Auto-ViT-Acc: An FPGA-Aware Automatic Acceleration Framework for Vision Transformer with Mixed-Scheme Quantization

arXiv 2208.05163 ↗First posted 10 Aug 2022cs.CV cs.LG eess.IVcited by 29

Not on the map. Why?

Abstract (arXiv)

Vision transformers (ViTs) are emerging with significantly improved accuracy in computer vision tasks. However, their complex architecture and enormous computation/storage demand impose urgent needs for new hardware accelerator design methodology. This work proposes an FPGA-aware automatic ViT acceleration framework based on the proposed mixed-scheme quantization. To the best of our knowledge, this is the first FPGA-based ViT acceleration framework exploring model quantization. Compared with state-of-the-art ViT quantization work (algorithmic approach only without hardware acceleration), our quantization achieves 0.47% to 1.36% higher Top-1 accuracy under the same bit-width. Compared with the 32-bit floating-point baseline FPGA accelerator, our accelerator achieves around 5.6x improvement on the frame rate (i.e., 56.8 FPS vs. 10.0 FPS) with 0.71% accuracy drop on ImageNet dataset for DeiT-base.

Key facts. Cited by 29 papers in the map (data through 14 Oct 2026). Builds on 22 of its 28 references that are in the map. First posted on arXiv 10 Aug 2022.

Builds on

22 of 28 references.

References · 22

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2020Training data-efficient image transformers & distillation through attentioncited by 4,586
  5. 2016Aggregated Residual Transformations for Deep Neural Networkscited by 2,546
  6. 2021Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNetcited by 1,101
  7. 2016XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networkscited by 849
  8. 2021Do Vision Transformers See Like Convolutional Neural Networks?cited by 675
  9. 2015BinaryConnect: Training Deep Neural Networks with binary weights during propagationscited by 592
  10. 2016DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradientscited by 569
  11. 2018PACT: Parameterized Clipping Activation for Quantized Neural Networkscited by 467
  12. 2018HAQ: Hardware-Aware Automated Quantization with Mixed Precisioncited by 443
10 more
  1. 2019HAWQ: Hessian AWare Quantization of Neural Networks with Mixed-Precisioncited by 402
  2. 2019Q8BERT: Quantized 8Bit BERTcited by 300
  3. 2021Post-Training Quantization for Vision Transformercited by 278
  4. 2020BinaryBERT: Pushing the Limit of BERT Quantizationcited by 178
  5. 2019Additive Powers-of-Two Quantization: An Efficient Non-uniform Discretization for Neural Networkscited by 148
  6. 2020TernaryBERT: Distillation-aware Ultra-low Bit BERTcited by 148
  7. 2017Extremely Low Bit Neural Network: Squeeze the Last Bit Out with ADMMcited by 61
  8. 2020FTRANS: Energy-Efficient Acceleration of Transformers using FPGAcited by 54
  9. 2020Mix and Match: A Novel FPGA-Centric Deep Neural Network Quantization Frameworkcited by 27
  10. 2018Simultaneously Optimizing Weight and Quantizer of Ternary Neural Network using Truncated Gaussian Approximationcited by 20

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 2; 2023: 5; 2024: 14; 2025: 4; 2026: 4.

29 papers.

Sort:

By area:

  1. 2024Efficient Multimodal Large Language Models: A Surveycited by 96
  2. 2022HeatViT: Hardware-Efficient Adaptive Token Pruning for Vision Transformerscited by 46
  3. 2024A Survey on Transformer Compressioncited by 46
  4. 2023A survey on efficient vision transformers: algorithms, techniques, and performance benchmarkingcited by 41
  5. 2025Vision Transformers on the Edge: A Comprehensive Survey of Model Compression and Acceleration Strategiescited by 24
  6. 2023Edge-MoE: Memory-Efficient Multi-Task Vision Transformer Architecture with Task-level Sparsity via Mixture-of-Expertscited by 23
  7. 2023Boost Vision Transformer with GPU-Friendly Sparsity and Quantizationcited by 20
  8. 2023Jumping through Local Minima: Quantization in the Loss Landscape of Vision Transformerscited by 18
  9. 2024Model Quantization and Hardware Acceleration for Vision Transformers: A Comprehensive Surveycited by 13
  10. 2023ViTA: A Vision Transformer Inference Accelerator for Edge Applicationscited by 11
  11. 2024HG-PIPE: Vision Transformer Acceleration with Hybrid-Grained Pipelinecited by 7
  12. 2025Pushing up to the Limit of Memory Bandwidth and Capacity Utilization for Efficient LLM Decoding on Embedded FPGAcited by 7
17 more
  1. 2024Accelerating ViT Inference on FPGA through Static and Dynamic Pruningcited by 5
  2. 2024An FPGA-Based Reconfigurable Accelerator for Convolution-Transformer Hybrid EfficientViTcited by 4
  3. 2024P2-ViT: Power-of-Two Post-Training Quantization and Acceleration for Fully Quantized Vision Transformercited by 4
  4. 2024Quasar-ViT: Hardware-Oriented Quantization-Aware Architecture Search for Vision Transformerscited by 4
  5. 2024Trio-ViT: Post-Training Quantization and Acceleration for Softmax-Free Efficient Vision Transformercited by 3
  6. 2024M2-ViT: Accelerating Hybrid Vision Transformers with Two-Level Mixed Quantizationcited by 3
  7. 2024TATAA: Programmable Mixed-Precision Transformer Acceleration with a Transformable Arithmetic Architecturecited by 3
  8. 2025LL-ViT: Edge Deployable Vision Transformers with Look Up Table Neuronscited by 3
  9. 2024CHOSEN: Compilation to Hardware Optimization Stack for Efficient Vision Transformer Inferencecited by 2
  10. 2024LUTMUL: Exceed Conventional FPGA Roofline Limit by LUT-based Efficient Multiplication for Neural Network Inferencecited by 2
  11. 2026TriCCOT: Tri-part Convolutional Conformal Transformer for Onboard Space Object Detectioncited by 2
  12. 2022CPT-V: A Contrastive Approach to Post-Training Quantization of Vision Transformerscited by 1
  13. 2024Resource-aware Mixed-precision Quantization for Enhancing Deployability of Transformers for Time-series Forecasting on Embedded FPGAscited by 1
  14. 2025FaRAccel: FPGA-Accelerated Defense Architecture for Efficient Bit-Flip Attack Resilience in Transformer Modelscited by 1
  15. 2026DAPA: Distribution Aware Piecewise Activation Functions for On-Device Transformer Inference and Trainingcited by 0
  16. 2026DxPTA: An Architecture Design Space Exploration with Optical Dataflow-guided Strategy for HW/SW Co-Design of Photonic Transformer Acceleratorscited by 0
  17. 2026FlexViT: A Flexible FPGA-based Accelerator for Edge Vision Transformerscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2022VAQF: Fully Automatic Software-Hardware Co-Design Framework for Low-Bit Vision Transformer
  2. 2025CoQMoE: Co-Designed Quantization and Computation Orchestration for Mixture-of-Experts Vision Transformer on FPGA
  3. 2022Q-ViT: Accurate and Fully Quantized Low-bit Vision Transformer
  4. 2025Refining Datapath for Microscaling ViTs
  5. 2024ViT-1.58b: Mobile Vision Transformers in the 1-bit Era
  6. 2024MPTQ-ViT: Mixed-Precision Post-Training Quantization for Vision Transformer
  7. 2025QUARK: Quantization-Enabled Circuit Sharing for Transformer Acceleration by Exploiting Common Patterns in Nonlinear Operations
  8. 2023Q-HyViT: Post-Training Quantization of Hybrid Vision Transformers with Bridge Block Reconstruction for IoT Systems