mlsanityData to 7 Oct 2026

Understanding vision transformer robustness through the lens of out-of-distribution detection

arXiv 2602.01459 ↗First posted 1 Feb 2026cs.CV cs.AI cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Vision transformers have shown remarkable performance in vision tasks, but enabling them for accessible and real-time use is still challenging. Quantization reduces memory and inference costs at the risk of performance loss. Strides have been made to mitigate low precision issues mainly by understanding in-distribution (ID) task behaviour, but the attention mechanism may provide insight on quantization attributes by exploring out-of-distribution (OOD) situations. We investigate the behaviour of quantized small-variant popular vision transformers (DeiT, DeiT3, and ViT) on common OOD datasets. ID analyses show the initial instabilities of 4-bit models, particularly of those trained on the larger ImageNet-22k, as the strongest FP32 model, DeiT3, sharply drop 17% from quantization error to be one of the weakest 4-bit models. While ViT shows reasonable quantization robustness for ID calibration, OOD detection reveals more: ViT and DeiT3 pretrained on ImageNet-22k respectively experienced a 15.0% and 19.2% average quantization delta in AUPR-out between full precision to 4-bit while their ImageNet-1k-only counterparts experienced a 9.5% and 12.0% delta. Overall, our results suggest pretraining on large scale datasets may hinder low-bit quantization robustness in OOD detection and that data augmentation may be a more beneficial option.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 17 of its 25 references that are in the map. First posted on arXiv 1 Feb 2026.

Builds on

17 of 25 references.

References · 17

  1. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  2. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,362
  3. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,548
  4. 2020Quantifying Attention Flow in Transformerscited by 759
  5. 2021How to train your ViT? Data, Augmentation, and Regularization in Vision Transformerscited by 562
  6. 2022ViM: Out-Of-Distribution with Virtual-logit Matchingcited by 404
  7. 2021Revisiting the Calibration of Modern Neural Networkscited by 387
  8. 2019Using Pre-Training Can Improve Model Robustness and Uncertaintycited by 373
  9. 2021PTQ4ViT: Post-training quantization for vision transformers with twin uniform quantizationcited by 173
  10. 2021FQ-ViT: Post-Training Quantization for Fully Quantized Vision Transformercited by 148
  11. 2022On The Computational Complexity of Self-Attentioncited by 137
  12. 2022RepQ-ViT: Scale Reparameterization for Post-Training Quantization of Vision Transformerscited by 134
5 more
  1. 2023Towards Accurate Post-Training Quantization for Vision Transformercited by 70
  2. 2022How Much Data Are Augmentations Worth? An Investigation into Scaling Laws, Invariance, and Implicit Regularizationcited by 40
  3. 2021Stability and Convergence of Stochastic Gradient Clipping: Beyond Lipschitz Continuity and Smoothnesscited by 37
  4. 2024RepQuant: Towards Accurate Post-Training Quantization of Large Transformer Models via Scale Reparameterizationcited by 10
  5. 2025Leveraging Registers in Vision Transformers for Robust Adaptationcited by 3

Built on by

Not cited yet.