mlsanityData to 14 Oct 2026

MixFrag: Fragility-Guided Mixed-Precision Post-Training Quantization for Vision Transformers

arXiv 2607.28589 ↗First posted 30 Jul 2026cs.CV cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Post-training quantization (PTQ) has emerged as an effective solution for deploying Vision Transformers (ViTs) on resource-constrained devices. However, existing PTQ methods typically employ uniform bit-widths across transformer components, overlooking their heterogeneous sensitivity to quantization and leading to inefficient precision allocation. In this paper, we propose {MixFrag, a fragility-guided mixed-precision PTQ framework for Vision Transformers. MixFrag first estimates component-level quantization fragility by measuring the Kullback--Leibler (KL) divergence between full-precision and isolated quantized output distributions using a small calibration set. It then formulates bit allocation as a Multiple-Choice Knapsack Problem (MCKP), enabling adaptive layer-wise precision assignment under a target bit budget. Extensive experiments on ImageNet-1K across multiple Vision Transformer architectures demonstrate that MixFrag achieves competitive classification performance under practical mixed-precision settings. Furthermore, evaluations on COCO object detection and instance segmentation show that MixFrag achieves state-of-the-art performance among existing mixed-precision PTQ methods, improving the previous best method by up to 9.6 AP under the challenging MP3/MP3 setting. Additional analyses validate the proposed fragility metric and demonstrate its strong correlation with the learned bit allocation. These results establish MixFrag as an effective framework for mixed-precision post-training quantization of Vision Transformers.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 45 of its 55 references that are in the map. First posted on arXiv 30 Jul 2026.

Builds on

45 of 55 references.

References · 45

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  4. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  5. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  6. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  7. 2017Mask R-CNNcited by 7,277
  8. 2020Training data-efficient image transformers & distillation through attentioncited by 4,586
  9. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,422
  10. 2017Cascade R-CNN: Delving into High Quality Object Detectioncited by 1,276
  11. 2021Segmenter: Transformer for Semantic Segmentationcited by 855
  12. 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 815
33 more
  1. 2020Up or Down? Adaptive Rounding for Post-Training Quantizationcited by 621
  2. 2021BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstructioncited by 479
  3. 2018PACT: Parameterized Clipping Activation for Quantized Neural Networkscited by 467
  4. 2018HAQ: Hardware-Aware Automated Quantization with Mixed Precisioncited by 443
  5. 2019HAWQ: Hessian AWare Quantization of Neural Networks with Mixed-Precisioncited by 402
  6. 2021Post-Training Quantization for Vision Transformercited by 278
  7. 2019HAWQ-V2: Hessian Aware trace-Weighted Quantization of Neural Networkscited by 261
  8. 2022QDrop: Randomly Dropping Quantization for Extremely Low-bit Post-Training Quantizationcited by 180
  9. 2021PTQ4ViT: Post-training quantization for vision transformers with twin uniform quantizationcited by 179
  10. 2021FQ-ViT: Post-Training Quantization for Fully Quantized Vision Transformercited by 151
  11. 2022RepQ-ViT: Scale Reparameterization for Post-Training Quantization of Vision Transformerscited by 137
  12. 2022Q-ViT: Accurate and Fully Quantized Low-bit Vision Transformercited by 108
  13. 2022I-ViT: Integer-only Quantization for Efficient Vision Transformer Inferencecited by 95
  14. 2022PD-Quant: Post-Training Quantization based on Prediction Difference Metriccited by 73
  15. 2023Towards Accurate Post-Training Quantization for Vision Transformercited by 71
  16. 2022DearKD: Data-Efficient Early Knowledge Distillation for Vision Transformerscited by 65
  17. 2024A Survey on Transformer Compressioncited by 46
  18. 2022CP-ViT: Cascade Vision Transformer Pruning via Progressive Sparsity Predictioncited by 39
  19. 2023OcTr: Octree-based Transformer for 3D Object Detectioncited by 32
  20. 2024PTQ4SAM: Post-Training Quantization for Segment Anythingcited by 32
  21. 2024AdaLog: Post-Training Quantization for Vision Transformers with Adaptive Logarithm Quantizercited by 29
  22. 2022BiViT: Extremely Compressed Binary Vision Transformercited by 28
  23. 2025Vision Transformers on the Edge: A Comprehensive Survey of Model Compression and Acceleration Strategiescited by 24
  24. 2024DopQ-ViT: Towards Distribution-Friendly and Outlier-Aware Post-Training Quantization for Vision Transformerscited by 19
  25. 2025FIMA-Q: Post-Training Quantization for Vision Transformers by Fisher Information Matrix Approximationcited by 19
  26. 2023Jumping through Local Minima: Quantization in the Loss Landscape of Vision Transformerscited by 18
  27. 2024Low-Rank Rescaled Vision Transformer Fine-Tuning: A Residual Design Approachcited by 18
  28. 2024Comprehensive Survey of Model Compression and Speed up for Vision Transformerscited by 14
  29. 2023Knowledge Distillation in Vision Transformers: A Critical Reviewcited by 13
  30. 2024Model Quantization and Hardware Acceleration for Vision Transformers: A Comprehensive Surveycited by 13
  31. 2025Mix-QViT: Mixed-Precision Vision Transformer Quantization Driven by Layer Importance and Quantization Sensitivitycited by 9
  32. 2023Compressing Vision Transformers for Low-Resource Visual Learningcited by 2
  33. 2025LampQ: Towards Accurate Layer-wise Mixed Precision Quantization for Vision Transformerscited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023Patch-wise Mixed-Precision Quantization of Vision Transformer
  2. 2025MixA-Q: Revisiting Activation Sparsity for Vision Transformers from a Mixed-Precision Quantization Perspective
  3. 2024MPTQ-ViT: Mixed-Precision Post-Training Quantization for Vision Transformer
  4. 2026Amortized-Precision Quantization for Early-Exit Vision Transformers
  5. 2024LRP-QViT: Mixed-Precision Vision Transformer Quantization via Layer-wise Relevance Propagation
  6. 2024P2-ViT: Power-of-Two Post-Training Quantization and Acceleration for Fully Quantized Vision Transformer
  7. 2023I&S-ViT: An Inclusive & Stable Method for Pushing the Limit of Post-Training ViTs Quantization
  8. 2026ScalePredictor: Instance-aware Scale Learning for Accurate Quantization of Vision Transformers