mlsanityData to 7 Oct 2026

Mix-QViT: Mixed-Precision Vision Transformer Quantization Driven by Layer Importance and Quantization Sensitivity

arXiv 2501.06357 ↗First posted 10 Jan 2025cs.CVcited by 9

Not on the map. Why?

Abstract (arXiv)

In this paper, we propose Mix-QViT, an explainability-driven MPQ framework that systematically allocates bit-widths to each layer based on two criteria: layer importance, assessed via Layer-wise Relevance Propagation (LRP), which identifies how much each layer contributes to the final classification, and quantization sensitivity, determined by evaluating the performance impact of quantizing each layer at various precision levels while keeping others layers at a baseline. Additionally, for post-training quantization (PTQ), we introduce a clipped channel-wise quantization method designed to reduce the effects of extreme outliers in post-LayerNorm activations by removing severe inter-channel variations. We validate our approach by applying Mix-QViT to ViT, DeiT, and Swin Transformer models across multiple datasets. Our experimental results for PTQ demonstrate that both fixed-bit and mixed-bit methods outperform existing techniques, particularly at 3-bit, 4-bit, and 6-bit precision. Furthermore, in quantization-aware training, Mix-QViT achieves superior performance with 2-bit mixed-precision.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 49 of its 52 references that are in the map. First posted on arXiv 10 Jan 2025.

Builds on

49 of 52 references.

References · 49

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  3. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  4. 2020End-to-End Object Detection with Transformerscited by 7,384
  5. 2017Mask R-CNNcited by 7,259
  6. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  7. 2016Gaussian Error Linear Units (GELUs)cited by 3,431
  8. 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,960
  9. 2020Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformerscited by 1,368
  10. 2017Cascade R-CNN: Delving into High Quality Object Detectioncited by 1,275
  11. 2021Segmenter: Transformer for Semantic Segmentationcited by 855
  12. 2021DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsificationcited by 849
37 more
  1. 2022Exploring Plain Vision Transformer Backbones for Object Detectioncited by 723
  2. 2016CVXPY: A Python-Embedded Modeling Language for Convex Optimizationcited by 633
  3. 2020Transformer Interpretability Beyond Attention Visualizationcited by 620
  4. 2019Learned Step Size Quantizationcited by 604
  5. 2018HAQ: Hardware-Aware Automated Quantization with Mixed Precisioncited by 426
  6. 2019HAWQ: Hessian AWare Quantization of Neural Networks with Mixed-Precisioncited by 387
  7. 2019Q-BERT: Hessian Based Ultra Low Precision Quantization of BERTcited by 355
  8. 2018Soft Filter Pruning for Accelerating Deep Convolutional Neural Networkscited by 313
  9. 2021Post-Training Quantization for Vision Transformercited by 274
  10. 2019HAWQ-V2: Hessian Aware trace-Weighted Quantization of Neural Networkscited by 250
  11. 2015Explaining NonLinear Classification Decisions with Deep Taylor Decompositioncited by 248
  12. 2022EfficientFormer: Vision Transformers at MobileNet Speedcited by 240
  13. 2022TinyViT: Fast Pretraining Distillation for Small Vision Transformerscited by 232
  14. 2013Predicting Parameters in Deep Learningcited by 223
  15. 2021Evo-ViT: Slow-Fast Token Evolution for Dynamic Vision Transformercited by 175
  16. 2021PTQ4ViT: Post-training quantization for vision transformers with twin uniform quantizationcited by 173
  17. 2021FQ-ViT: Post-Training Quantization for Fully Quantized Vision Transformercited by 148
  18. 2022RepQ-ViT: Scale Reparameterization for Post-Training Quantization of Vision Transformerscited by 134
  19. 2022Q-ViT: Accurate and Fully Quantized Low-bit Vision Transformercited by 106
  20. 2022MiniViT: Compressing Vision Transformers with Weight Multiplexingcited by 88
  21. 2022Knowledge Distillation via the Target-aware Transformercited by 70
  22. 2023Towards Accurate Post-Training Quantization for Vision Transformercited by 70
  23. 2021Global Vision Transformer Pruning with Hessian-Aware Saliencycited by 69
  24. 2022DearKD: Data-Efficient Early Knowledge Distillation for Vision Transformerscited by 65
  25. 2019AutoQ: Automated Kernel-Wise Neural Network Quantizationcited by 60
  26. 2021Towards Mixed-Precision Quantization of Neural Networks via Constrained Optimizationcited by 57
  27. 2022Patch Similarity Aware Data-Free Quantization for Vision Transformerscited by 55
  28. 2023Oscillation-free Quantization for Low-bit Vision Transformerscited by 55
  29. 2022Mixed-Precision Neural Network Quantization via Learned Layer-wise Importancecited by 53
  30. 2022Q-ViT: Fully Differentiable Quantization for Vision Transformercited by 36
  31. 2024AdaLog: Post-Training Quantization for Vision Transformers with Adaptive Logarithm Quantizercited by 28
  32. 2023Quantization Variation: A New Perspective on Training Transformers with Low-Bit Precisioncited by 17
  33. 2023Patch-wise Mixed-Precision Quantization of Vision Transformercited by 15
  34. 2021Mixed Precision of Quantization of Transformer Language Models for Speech Recognitioncited by 10
  35. 2024MPTQ-ViT: Mixed-Precision Post-Training Quantization for Vision Transformercited by 10
  36. 2023TSPTQ-ViT: Two-scaled post-training quantization for vision transformercited by 7
  37. 2023Mixed-Precision Quantization for Deep Vision Models with Integer Quadratic Programmingcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 4; 2026: 5.

9 papers.

  1. 2026Locate, Steer, and Improve: A Practical Survey of Actionable Mechanistic Interpretability in Large Language Modelscited by 22
  2. 2025Mix-QSAM: Mixed-Precision Quantization of the Segment Anything Modelcited by 4
  3. 2025MSQ: Memory-Efficient Bit Sparsification Quantizationcited by 2
  4. 2026Quantizing Time-Series Models As Dynamical Systems: Trajectory-Based Quantization Sensitivity Scorecited by 2
  5. 2025Exploiting Information Redundancy in Attention Maps for Extreme Quantization of Vision Transformerscited by 1
  6. 2025I-Segmenter: Integer-Only Vision Transformer for Efficient Semantic Segmentationcited by 1
  7. 2026SFMP: Fine-Grained, Hardware-Friendly and Search-Free Mixed-Precision Quantization for Large Language Modelscited by 1
  8. 2026MixFrag: Fragility-Guided Mixed-Precision Post-Training Quantization for Vision Transformerscited by 0
  9. 2026Teacher-Anchored Selection of Post-Training Quantized Models under Domain Shiftcited by 0