mlsanityData to 14 Oct 2026

DeVIT: Low-Power Vision Transformer Acceleration Using Delta Computation

arXiv 2608.01343 ↗First posted 2 Aug 2026cs.CV cs.AI cs.ARcited by 0

Not on the map. Why?

Abstract (arXiv)

The emergence of transformer-based deep learning models has brought unprecedented performance across various domains, particularly in natural language processing and computer vision. However, deploying these models, especially on resource-constrained devices, poses significant challenges due to their high computational complexity and large memory size and bandwidth requirements. This complexity has led researchers to use low-bit model weights to reduce memory usage and improve efficiency. In addition to reducing processing and memory demands, quantization introduces another useful property: value locality, where the extremely large number of parameters are restricted to a limited range of values. To fully take advantage of this locality, this paper presents DeVIT, an acceleration method for vision transformers that leverages differential computation to enable multiplier-less matrix multiplication.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 26 of its 38 references that are in the map. First posted on arXiv 2 Aug 2026.

Builds on

26 of 38 references.

References · 26

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  2. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  3. 2020End-to-End Object Detection with Transformerscited by 7,414
  4. 2020Training data-efficient image transformers & distillation through attentioncited by 4,586
  5. 2016Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Networkcited by 1,908
  6. 2022Token Merging: Your ViT But Fastercited by 1,021
  7. 2021DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsificationcited by 864
  8. 2021PTQ4ViT: Post-training quantization for vision transformers with twin uniform quantizationcited by 179
  9. 2019Additive Powers-of-Two Quantization: An Efficient Non-uniform Discretization for Neural Networkscited by 148
  10. 2022I-ViT: Integer-only Quantization for Efficient Vision Transformer Inferencecited by 95
  11. 2020ShiftAddNet: A Hardware-Inspired Deep Networkcited by 52
  12. 2022HeatViT: Hardware-Efficient Adaptive Token Pruning for Vision Transformerscited by 46
14 more
  1. 2022ViTCoD: Vision Transformer Acceleration via Dedicated Algorithm and Accelerator Co-Designcited by 41
  2. 2022Q-ViT: Fully Differentiable Quantization for Vision Transformercited by 37
  3. 2024ShiftAddLLM: Accelerating Pretrained LLMs via Post-Training Multiplication-Less Reparameterizationcited by 30
  4. 2022An Algorithm-Hardware Co-Optimized Framework for Accelerating N:M Sparse Transformerscited by 29
  5. 2023AccelTran: A Sparsity-Aware Accelerator for Dynamic Inference with Transformerscited by 24
  6. 2025Similarity-Guided Layer-Adaptive Vision Transformer for UAV Trackingcited by 23
  7. 2018UCNN: Exploiting Computational Reuse in Deep Neural Networks via Weight Repetitioncited by 20
  8. 2023SwiftTron: An Efficient Hardware Accelerator for Quantized Transformerscited by 18
  9. 2024Progressive Gradient Flow for Robust N:M Sparsity Training in Transformerscited by 18
  10. 2024LUT Tensor Core: A Software-Hardware Co-Design for LUT-Based Low-Bit LLM Inferencecited by 18
  11. 2023ShiftAddViT: Mixture of Multiplication Primitives Towards Efficient Vision Transformercited by 15
  12. 2025RePaViT: Scalable Vision Transformer Acceleration via Structural Reparameterization on Feedforward Network Layerscited by 5
  13. 2024NU-Class Net: A Novel Approach for Video Quality Enhancementcited by 3
  14. 2025Low Power Vision Transformer Accelerator with Hardware-Aware Pruning and Optimized Dataflowcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Model Quantization and Hardware Acceleration for Vision Transformers: A Comprehensive Survey
  2. 2024Trio-ViT: Post-Training Quantization and Acceleration for Softmax-Free Efficient Vision Transformer
  3. 2024P2-ViT: Power-of-Two Post-Training Quantization and Acceleration for Fully Quantized Vision Transformer
  4. 2023A survey on efficient vision transformers: algorithms, techniques, and performance benchmarking
  5. 2022Q-ViT: Accurate and Fully Quantized Low-bit Vision Transformer
  6. 2025Low-Bit Integerization of Vision Transformers using Operand Reordering for Efficient Hardware
  7. 2024ViT-1.58b: Mobile Vision Transformers in the 1-bit Era
  8. 2022SaiT: Sparse Vision Transformers through Adaptive Token Pruning