mlsanityData to 7 Oct 2026

DopQ-ViT: Towards Distribution-Friendly and Outlier-Aware Post-Training Quantization for Vision Transformers

arXiv 2408.03291 ↗First posted 6 Aug 2024cs.CVcited by 19

Not on the map. Why?

Abstract (arXiv)

Vision Transformers (ViTs) have gained significant attention, but their high computing cost limits the practical applications. While post-training quantization (PTQ) reduces model size and speeds up inference, it often degrades performance, especially in low-bit settings. We identify two key reasons for the performance degradation: 1) existing quantization methods fail to align with the power-law distribution of post-Softmax activations, and 2) reparameterizing post-LayerNorm activations leads to a performance drop due to the significant influence of outliers in the scaling factors. To address these challenges, we propose DopQ-ViT, a Distribution-friendly and Outlier-aware Post-training Quantization method for ViTs. First, DopQ-ViT introduces the Tan Quantizer (TanQ), which better preserves the power-law distribution of post-Softmax activations by focusing more on values near 1. Second, DopQ-ViT presents the MAD-guided Optimal Scaling Factor (MOSF), which selects the optimal scaling factor without introducing additional calculations. Extensive experiments across various ViT models and quantization settings demonstrate that DopQ-ViT, with the help of TanQ and MOSF, outperforms previous PTQ methods on both classification and detection tasks.

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). Builds on 46 of its 61 references that are in the map. First posted on arXiv 6 Aug 2024.

Builds on

46 of 61 references.

References · 46

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  5. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  6. 2020End-to-End Object Detection with Transformerscited by 7,384
  7. 2017Mask R-CNNcited by 7,259
  8. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  9. 2017Cascade R-CNN: Delving into High Quality Object Detectioncited by 1,275
  10. 2021Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNetcited by 1,096
  11. 2021CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classificationcited by 806
  12. 2022Exploring Plain Vision Transformer Backbones for Object Detectioncited by 723
34 more
  1. 2019Learned Step Size Quantizationcited by 604
  2. 2018Quantizing deep convolutional networks for efficient inference: A whitepapercited by 507
  3. 2021BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstructioncited by 464
  4. 2018PACT: Parameterized Clipping Activation for Quantized Neural Networkscited by 456
  5. 2020End-to-End Video Instance Segmentation with Transformerscited by 443
  6. 2023OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Modelscited by 403
  7. 2021Post-Training Quantization for Vision Transformercited by 274
  8. 2021Understanding Robustness of Transformers for Image Classificationcited by 250
  9. 2019Differentiable Soft Quantization: Bridging Full-Precision and Low-Bit Neural Networkscited by 234
  10. 2022TinyViT: Fast Pretraining Distillation for Small Vision Transformerscited by 232
  11. 2022Outlier Suppression: Pushing the Limit of Low-bit Transformer Language Modelscited by 176
  12. 2022QDrop: Randomly Dropping Quantization for Extremely Low-bit Post-Training Quantizationcited by 175
  13. 2021PTQ4ViT: Post-training quantization for vision transformers with twin uniform quantizationcited by 173
  14. 2024DuQuant: Distributing Outliers via Dual Transformation Makes Stronger Quantized LLMscited by 165
  15. 2021FQ-ViT: Post-Training Quantization for Fully Quantized Vision Transformercited by 148
  16. 2022RepQ-ViT: Scale Reparameterization for Post-Training Quantization of Vision Transformerscited by 134
  17. 2022Overcoming Oscillations in Quantization-Aware Trainingcited by 116
  18. 2022Q-ViT: Accurate and Fully Quantized Low-bit Vision Transformercited by 106
  19. 2022I-ViT: Integer-only Quantization for Efficient Vision Transformer Inferencecited by 93
  20. 2021Vision Transformer Pruningcited by 77
  21. 2022PD-Quant: Post-Training Quantization based on Prediction Difference Metriccited by 73
  22. 2023Towards Accurate Post-Training Quantization for Vision Transformercited by 70
  23. 2022Patch Similarity Aware Data-Free Quantization for Vision Transformerscited by 55
  24. 2023Oscillation-free Quantization for Low-bit Vision Transformerscited by 55
  25. 2022PSAQ-ViT V2: Towards Accurate and General Data-Free Quantization for Vision Transformerscited by 42
  26. 2022Q-ViT: Fully Differentiable Quantization for Vision Transformercited by 36
  27. 2024MoPE-CLIP: Structured Pruning for Efficient Vision-Language Models with Module-wise Pruning Error Metriccited by 30
  28. 2024Towards Accurate Post-Training Quantization of Vision Transformers via Error Reductioncited by 23
  29. 2022Dynamic Dual Trainable Bounds for Ultra-low Precision Super-Resolution Networkscited by 20
  30. 2023I&S-ViT: An Inclusive & Stable Method for Pushing the Limit of Post-Training ViTs Quantizationcited by 19
  31. 2023Jumping through Local Minima: Quantization in the Loss Landscape of Vision Transformerscited by 18
  32. 2024RepQuant: Towards Accurate Post-Training Quantization of Large Transformer Models via Scale Reparameterizationcited by 10
  33. 2023MBQuant: A Novel Multi-Branch Topology Method for Arbitrary Bit-width Network Quantizationcited by 6
  34. 2024MGRQ: Post-Training Quantization For Vision Transformer With Mixed Granularity Reconstructioncited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 10; 2026: 9.

19 papers.

Sort:

By area:

  1. 2025Quantization Meets dLLMs: A Systematic Study of Post-training Quantization for Diffusion LLMscited by 21
  2. 2025APHQ-ViT: Post-Training Quantization with Average Perturbation Hessian Based Reconstruction for Vision Transformerscited by 17
  3. 2025FIMA-Q: Post-Training Quantization for Vision Transformers by Fisher Information Matrix Approximationcited by 16
  4. 2025Quantized Visual Geometry Grounded Transformercited by 16
  5. 2025Low-bit Model Quantization for Deep Neural Networks: A Surveycited by 11
  6. 2025LRQ-DiT: Log-Rotation Post-Training Quantization of Diffusion Transformers for Image and Video Generationcited by 10
  7. 2026DuQuant++: Fine-grained Rotation Enhances Microscaling FP4 Quantizationcited by 8
  8. 2025OuroMamba: A Data-Free Quantization Framework for Vision Mambacited by 5
  9. 2025MedREK: Retrieval-Based Editing for Medical LLMs with Key-Aware Promptscited by 5
  10. 2025AHCQ-SAM: Toward Accurate and Hardware-Compatible Post-Training Segment Anything Model Quantizationcited by 3
  11. 2026Benchmarking Trustworthiness of SLMs: Pre-trained vs. Compressedcited by 2
  12. 2025Activation Quantization of Vision Encoders Needs Prefixing Registerscited by 1
7 more
  1. 2026Colinearity Decay: Training Quantization-Friendly ViTs with Outlier Decaycited by 1
  2. 2026Nonlinear Bipolar Compensation: Handling Outliers in Post-Training Quantizationcited by 1
  3. 2026QVGGT: Post-Training Quantized Visual Geometry Grounded Transformercited by 1
  4. 2026FTerViT: Fully Ternary Vision Transformercited by 0
  5. 2026MixFrag: Fragility-Guided Mixed-Precision Post-Training Quantization for Vision Transformerscited by 0
  6. 2026Channel-Wise and Token-Aware Post-Training Quantization for Visual State Space Dualitycited by 0
  7. 2026GLF-Q: Global-Local Feature-based Quantization for Vision Transformerscited by 0