mlsanityData to 14 Oct 2026

GLF-Q: Global-Local Feature-based Quantization for Vision Transformers

arXiv 2609.34564 ↗First posted 28 Sep 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Post-training quantization (PTQ) efficiently compresses Vision Transformers (ViTs) without retraining, yet suffers severe accuracy degradation at low bit-widths. Existing optimization-based PTQ methods guide block reconstruction via either soft logits or second-order Hessian proxies. Logit supervision is prone to overfitting on limited calibration data, while Hessian approximations incur structural truncation errors. To address these limitations, we propose GLF-Q, a novel PTQ framework guided by Global-Local Feature alignment. GLF-Q propagates quantized block outputs through downstream full-precision layers to align penultimate-layer representations under local output regularization, providing downstream feature supervision without explicitly approximating the Hessian or using a Taylor expansion. Furthermore, offline Hadamard transformations are introduced with zero runtime overhead to disperse activation outliers across channels, effectively contracting dynamic ranges and reducing quantization errors. Meanwhile, optimizing this loss via a Straight-Through Estimator (STE) achieves rapid convergence, bypassing continuous relaxation rounding formulations such as AdaRound. Extensive experiments across representative ViT architectures demonstrate that GLF-Q with standard uniform quantizers substantially outperforms state-of-the-art methods under 3-bit quantization on image classification. In addition, GLF-Q exhibits strong out-of-domain calibration robustness and achieves speedups under 8-bit GPU deployment.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 31 of its 37 references that are in the map. First posted on arXiv 28 Sep 2026.

Builds on

31 of 37 references.

References · 31

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  3. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  4. 2017Mask R-CNNcited by 7,277
  5. 2020Training data-efficient image transformers & distillation through attentioncited by 4,586
  6. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,079
  7. 2019MMDetection: Open MMLab Detection Toolbox and Benchmarkcited by 1,385
  8. 2017Cascade R-CNN: Delving into High Quality Object Detectioncited by 1,276
  9. 2023Vision Transformers Need Registerscited by 871
  10. 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 815
  11. 2024QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMscited by 634
  12. 2019Learned Step Size Quantizationcited by 628
19 more
  1. 2020Up or Down? Adaptive Rounding for Post-Training Quantizationcited by 621
  2. 2021BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstructioncited by 479
  3. 2024SliceGPT: Compress Large Language Models by Deleting Rows and Columnscited by 385
  4. 2022QDrop: Randomly Dropping Quantization for Extremely Low-bit Post-Training Quantizationcited by 180
  5. 2021PTQ4ViT: Post-training quantization for vision transformers with twin uniform quantizationcited by 179
  6. 2022RepQ-ViT: Scale Reparameterization for Post-Training Quantization of Vision Transformerscited by 137
  7. 2022Q-ViT: Accurate and Fully Quantized Low-bit Vision Transformercited by 108
  8. 2022PD-Quant: Post-Training Quantization based on Prediction Difference Metriccited by 73
  9. 2023Towards Accurate Post-Training Quantization for Vision Transformercited by 71
  10. 2020Distilling Knowledge by Mimicking Featurescited by 35
  11. 2024AdaLog: Post-Training Quantization for Vision Transformers with Adaptive Logarithm Quantizercited by 29
  12. 2024Instance-Aware Group Quantization for Vision Transformerscited by 25
  13. 2024Quantization without Tearscited by 25
  14. 2023I&S-ViT: An Inclusive & Stable Method for Pushing the Limit of Post-Training ViTs Quantizationcited by 19
  15. 2024DopQ-ViT: Towards Distribution-Friendly and Outlier-Aware Post-Training Quantization for Vision Transformerscited by 19
  16. 2025APHQ-ViT: Post-Training Quantization with Average Perturbation Hessian Based Reconstruction for Vision Transformerscited by 19
  17. 2025FIMA-Q: Post-Training Quantization for Vision Transformers by Fisher Information Matrix Approximationcited by 19
  18. 2025GPLQ: A General, Practical, and Lightning QAT Method for Vision Transformerscited by 8
  19. 2026Nonlinear Bipolar Compensation: Handling Outliers in Post-Training Quantizationcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024MGRQ: Post-Training Quantization For Vision Transformer With Mixed Granularity Reconstruction
  2. 2021FQ-ViT: Post-Training Quantization for Fully Quantized Vision Transformer
  3. 2024ADFQ-ViT: Activation-Distribution-Friendly Post-Training Quantization for Vision Transformers
  4. 2024MPTQ-ViT: Mixed-Precision Post-Training Quantization for Vision Transformer
  5. 2024Progressive Fine-to-Coarse Reconstruction for Accurate Low-Bit Post-Training Quantization in Vision Transformers
  6. 2026TR-PTQ: High-Accuracy Integer-Only Transformer Post Training Quantization via Taylor Region Reformulation
  7. 2025LampQ: Towards Accurate Layer-wise Mixed Precision Quantization for Vision Transformers
  8. 2025AIQViT: Architecture-Informed Post-Training Quantization for Vision Transformers