mlsanityData to 14 Oct 2026

Amortized-Precision Quantization for Early-Exit Vision Transformers

arXiv 2605.07317 ↗First posted 8 May 2026cs.CV cs.AIcited by 0

On the map: Efficient LLM inference and serving → Token pruning and merging for ViTs

Abstract (arXiv)

Vision Transformers (ViTs) achieve strong performance across vision tasks, yet their deployment with low-precision early exiting remains fragile. Existing quantization methods assume static full-depth execution, making them unstable when exit decisions are perturbed by quantization noise, which can amplify errors along dynamic inference paths. In this paper, we introduce Amortized-Precision Quantization (APQ), a utilization-aware formulation that accounts for layer-wise stochastic exposure to quantization noise and reveals depth-precision trade-offs. Building on APQ, we propose Mutual Adaptive Quantization with Early Exiting (MAQEE), a bi-level framework that jointly optimizes exit thresholds and bit-widths under explicit risk control to improve inference stability. MAQEE establishes a superior Pareto frontier in the accuracy-efficiency trade-off, reducing BOPs by up to 95% while maintaining accuracy and outperforming strong baselines by up to 20% across classification, detection, and segmentation tasks.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Token pruning and merging for ViTs, one of 12 directions in Efficient LLM inference and serving. Builds on 39 of its 49 references that are in the map. First posted on arXiv 8 May 2026.

Builds on

39 of 49 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023LGViT: Dynamic Early Exiting for Accelerating Vision Transformer1 Aug

Other references · 38

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  4. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  5. 2020End-to-End Object Detection with Transformerscited by 7,414
  6. 2020Training data-efficient image transformers & distillation through attentioncited by 4,586
  7. 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,940
  8. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,593
  9. 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 815
  10. 2021CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classificationcited by 806
  11. 2021Scaling Vision with Sparse Mixture of Expertscited by 747
  12. 2021Do Vision Transformers See Like Convolutional Neural Networks?cited by 675
26 more
  1. 2022Mixture-of-Experts with Expert Choice Routingcited by 623
  2. 2017BranchyNet: Fast Inference via Early Exiting from Deep Neural Networkscited by 596
  3. 2015BinaryConnect: Training Deep Neural Networks with binary weights during propagationscited by 592
  4. 2018Quantizing deep convolutional networks for efficient inference: A whitepapercited by 521
  5. 2020DeeBERT: Dynamic Early Exiting for Accelerating BERT Inferencecited by 332
  6. 2022Confident Adaptive Language Modelingcited by 310
  7. 2024LayerSkip: Enabling Early Exit Inference and Self-Speculative Decodingcited by 307
  8. 2021Post-Training Quantization for Vision Transformercited by 278
  9. 2022Tutel: Adaptive Mixture-of-Experts at Scalecited by 201
  10. 2021You Only Look at One Sequence: Rethinking Transformer in Vision through Object Detectioncited by 170
  11. 2019Quantization Networkscited by 153
  12. 2022RepQ-ViT: Scale Reparameterization for Post-Training Quantization of Vision Transformerscited by 137
  13. 2022Overcoming Oscillations in Quantization-Aware Trainingcited by 119
  14. 2021UTNet: A Hybrid Transformer Architecture for Medical Image Segmentationcited by 116
  15. 2022Q-ViT: Accurate and Fully Quantized Low-bit Vision Transformercited by 108
  16. 2021Adaptive Inference through Early-Exit Networks: Design, Challenges and Directionscited by 75
  17. 2023Towards Accurate Post-Training Quantization for Vision Transformercited by 71
  18. 2025MoEQuant: Enhancing Quantization for Mixture-of-Experts Large Language Models via Expert-Balanced Sampling and Affinity Guidancecited by 34
  19. 2024Fast yet Safe: Early-Exiting with Risk Controlcited by 30
  20. 2021Multi-Exit Vision Transformer for Dynamic Inferencecited by 28
  21. 2021Sliced Recursive Transformercited by 26
  22. 2024Towards Accurate Post-Training Quantization of Vision Transformers via Error Reductioncited by 23
  23. 2023Patch-wise Mixed-Precision Quantization of Vision Transformercited by 15
  24. 2024MPTQ-ViT: Mixed-Precision Post-Training Quantization for Vision Transformercited by 10
  25. 2024P2-ViT: Power-of-Two Post-Training Quantization and Acceleration for Fully Quantized Vision Transformercited by 4
  26. 2024Predicting Probabilities of Error to Combine Quantization and Early Exiting: QuEEcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024ADFQ-ViT: Activation-Distribution-Friendly Post-Training Quantization for Vision Transformers
  2. 2025LampQ: Towards Accurate Layer-wise Mixed Precision Quantization for Vision Transformers
  3. 2026MixFrag: Fragility-Guided Mixed-Precision Post-Training Quantization for Vision Transformers
  4. 2024DopQ-ViT: Towards Distribution-Friendly and Outlier-Aware Post-Training Quantization for Vision Transformers
  5. 2026Colinearity Decay: Training Quantization-Friendly ViTs with Outlier Decay
  6. 2024AdaLog: Post-Training Quantization for Vision Transformers with Adaptive Logarithm Quantizer
  7. 2024ViT-1.58b: Mobile Vision Transformers in the 1-bit Era
  8. 2023I&S-ViT: An Inclusive & Stable Method for Pushing the Limit of Post-Training ViTs Quantization