mlsanityData to 14 Oct 2026

FIMA-Q: Post-Training Quantization for Vision Transformers by Fisher Information Matrix Approximation

arXiv 2506.11543 ↗First posted 13 Jun 2025cs.CV cs.AI cs.LGcited by 19

Not on the map. Why?

Abstract (arXiv)

Post-training quantization (PTQ) has stood out as a cost-effective and promising model compression paradigm in recent years, as it avoids computationally intensive model retraining. Nevertheless, current PTQ methods for Vision Transformers (ViTs) still suffer from significant accuracy degradation, especially under low-bit quantization. To address these shortcomings, we analyze the prevailing Hessian-guided quantization loss, and uncover certain limitations of conventional Hessian approximations. By following the block-wise reconstruction framework, we propose a novel PTQ method for ViTs, dubbed FIMA-Q. Specifically, we firstly establish the connection between KL divergence and FIM, which enables fast computation of the quantization loss during reconstruction. We further propose an efficient FIM approximation method, namely DPLR-FIM, by employing the diagonal plus low-rank principle, and formulate the ultimate quantization loss. Our extensive experiments, conducted across various vision tasks with representative ViT-based architectures on public datasets, demonstrate that our method substantially promotes the accuracy compared to the state-of-the-art approaches, especially in the case of low-bit quantization. The source code is available at https://github.com/ShiheWang/FIMA-Q.

Key facts. Cited by 19 papers in the map (data through 14 Oct 2026). Builds on 31 of its 38 references that are in the map. First posted on arXiv 13 Jun 2025.

Builds on

31 of 38 references.

References · 31

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  5. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  6. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  7. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  8. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  9. 2017Mask R-CNNcited by 7,277
  10. 2020Training data-efficient image transformers & distillation through attentioncited by 4,586
  11. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,422
  12. 2019MMDetection: Open MMLab Detection Toolbox and Benchmarkcited by 1,385
19 more
  1. 2017Cascade R-CNN: Delving into High Quality Object Detectioncited by 1,276
  2. 2019Learned Step Size Quantizationcited by 628
  3. 2020Up or Down? Adaptive Rounding for Post-Training Quantizationcited by 621
  4. 2018Quantizing deep convolutional networks for efficient inference: A whitepapercited by 521
  5. 2021BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstructioncited by 479
  6. 2022QDrop: Randomly Dropping Quantization for Extremely Low-bit Post-Training Quantizationcited by 180
  7. 2021PTQ4ViT: Post-training quantization for vision transformers with twin uniform quantizationcited by 179
  8. 2021FQ-ViT: Post-Training Quantization for Fully Quantized Vision Transformercited by 151
  9. 2021Understanding and Overcoming the Challenges of Efficient Transformer Quantizationcited by 141
  10. 2022RepQ-ViT: Scale Reparameterization for Post-Training Quantization of Vision Transformerscited by 137
  11. 2022Q-ViT: Accurate and Fully Quantized Low-bit Vision Transformercited by 108
  12. 2022I-ViT: Integer-only Quantization for Efficient Vision Transformer Inferencecited by 95
  13. 2023Towards Accurate Post-Training Quantization for Vision Transformercited by 71
  14. 2022NoisyQuant: Noisy Bias-Enhanced Post-Training Activation Quantization for Vision Transformerscited by 61
  15. 2023OcTr: Octree-based Transformer for 3D Object Detectioncited by 32
  16. 2024AdaLog: Post-Training Quantization for Vision Transformers with Adaptive Logarithm Quantizercited by 29
  17. 2024Instance-Aware Group Quantization for Vision Transformerscited by 25
  18. 2023I&S-ViT: An Inclusive & Stable Method for Pushing the Limit of Post-Training ViTs Quantizationcited by 19
  19. 2024DopQ-ViT: Towards Distribution-Friendly and Outlier-Aware Post-Training Quantization for Vision Transformerscited by 19

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 16.

19 papers.

Sort:

By area:

  1. 2026SGMD: Score Gradient Matching Distillation for Few-Step Video Diffusion Distillationcited by 6
  2. 2025TreeQ: Pushing the Quantization Boundary of Diffusion Transformer via Tree-Structured Mixed-Precision Searchcited by 3
  3. 2025Activation Quantization of Vision Encoders Needs Prefixing Registerscited by 2
  4. 2025I-Segmenter: Integer-Only Vision Transformer for Efficient Semantic Segmentationcited by 1
  5. 2026Joint Post-Training Quantization of Vision Transformers with Learned Prompt-Guided Data Generationcited by 1
  6. 2026Colinearity Decay: Training Quantization-Friendly ViTs with Outlier Decaycited by 1
  7. 2026Nonlinear Bipolar Compensation: Handling Outliers in Post-Training Quantizationcited by 1
  8. 2026Revisiting Adaptive Rounding with Vectorized Reparameterization for LLM Quantizationcited by 0
  9. 2026When W4A4 Breaks Camouflaged Object Detection: Token-Group Dual-Constraint Activation Quantizationcited by 0
  10. 2026MARR: Module-Adaptive Residual Reconstruction for Low-Bit Post-Training Quantizationcited by 0
  11. 2026ScalePredictor: Instance-aware Scale Learning for Accurate Quantization of Vision Transformerscited by 0
  12. 2026MixFrag: Fragility-Guided Mixed-Precision Post-Training Quantization for Vision Transformerscited by 0
7 more
  1. 2026QuaSAR: Quantization Compensation via Stable Activation-Aware Rank Truncationcited by 0
  2. 2026SandwichQuant: Which Parameters Matter Before and After Quantization?cited by 0
  3. 2026Channel-Wise and Token-Aware Post-Training Quantization for Visual State Space Dualitycited by 0
  4. 2026GLF-Q: Global-Local Feature-based Quantization for Vision Transformerscited by 0
  5. 2026VisionMX: Unlocking Microscaling Post-Training Quantization for Vision Modelscited by 0
  6. 2026RPFQ-ViT: Rotated Phase-Frame Quantization for Extremely Low-Bit Weights in Vision Transformerscited by 0
  7. 2026Hardware-Friendly Token-Group Activation Quantization for Low-Bit Mamba Super-Resolutioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025APHQ-ViT: Post-Training Quantization with Average Perturbation Hessian Based Reconstruction for Vision Transformers
  2. 2024MPTQ-ViT: Mixed-Precision Post-Training Quantization for Vision Transformer
  3. 2024Progressive Fine-to-Coarse Reconstruction for Accurate Low-Bit Post-Training Quantization in Vision Transformers
  4. 2024MGRQ: Post-Training Quantization For Vision Transformer With Mixed Granularity Reconstruction
  5. 2025IPTQ-ViT: Post-Training Quantization of Non-linear Functions for Integer-only Vision Transformers
  6. 2025AIQViT: Architecture-Informed Post-Training Quantization for Vision Transformers
  7. 2023TSPTQ-ViT: Two-scaled post-training quantization for vision transformer
  8. 2024ADFQ-ViT: Activation-Distribution-Friendly Post-Training Quantization for Vision Transformers