mlsanityData to 14 Oct 2026

Less Precise Can Be More Reliable: A Systematic Evaluation of Quantization's Impact on VLMs Beyond Accuracy

arXiv 2509.21173 ↗First posted 25 Sep 2025cs.CV cs.AI cs.LGcited by 3

On the map: Multimodal LLMs / vision-language models → Visual token pruning and efficiency

Abstract (arXiv)

Vision-Language Models (VLMs) such as CLIP have revolutionized zero-shot classification and safety-critical tasks, including Out-of-Distribution (OOD) detection. However, their high computational cost hinders efficient real-world deployment. While quantization is a standard solution for efficiency, its broader impact on reliability metrics beyond simple Top-1 accuracy remains critically under-explored. In this study, we conduct a large-scale evaluation of VLM quantization across a comprehensive experimental suite of over 700k evaluation runs with varying configurations. We find that, contrary to the assumption that quantization's noise degrades performance, it can simultaneously improve accuracy, calibration, OOD detection, and robustness to noise, though not to covariate shift or spurious correlations. We leverage these counterintuitive findings to characterize the mechanics of quantization beyond simple regularization: we show that quantization dampens high-rank spectral components, compelling the model to rely more heavily on robust, low-rank features. Ultimately, this spectral filtering effect drives the observed improvements in generalization and noise tolerance, establishing a pathway to deploy faster, more reliable VLMs by utilizing quantization beyond its conventional role.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Visual token pruning and efficiency, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 57 of its 67 references that are in the map. First posted on arXiv 25 Sep 2025.

Builds on

57 of 67 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Q-VLM: Post-training Quantization for Large Vision-Language Models10 Oct

Other references · 56

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2017On Calibration of Modern Neural Networkscited by 4,960
  3. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
  4. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,864
  5. 2022A ConvNet for the 2020scited by 3,579
  6. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
  7. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,877
  8. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,385
  9. 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,850
  10. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,593
  11. 2019Do ImageNet Classifiers Generalize to ImageNet?cited by 1,515
  12. 2018A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attackscited by 1,485
38 more
  1. 2019Natural Adversarial Examplescited by 1,408
  2. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
  3. 2018On the Spectral Bias of Neural Networkscited by 1,341
  4. 2020Energy-based Out-of-distribution Detectioncited by 1,299
  5. 2022CoCa: Contrastive Captioners are Image-Text Foundation Modelscited by 1,274
  6. 2019Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalizationcited by 1,247
  7. 2019Learning Robust Global Representations by Penalizing Local Predictive Powercited by 1,013
  8. 2023Vision Transformers Need Registerscited by 871
  9. 2021Generalized Out-of-Distribution Detection: A Surveycited by 802
  10. 2022Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distributioncited by 739
  11. 2024QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMscited by 634
  12. 2019Learned Step Size Quantizationcited by 628
  13. 2015BinaryConnect: Training Deep Neural Networks with binary weights during propagationscited by 592
  14. 2023OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Modelscited by 424
  15. 2019Frequency Principle: Fourier Analysis Sheds Light on Deep Neural Networkscited by 368
  16. 2021Multimodal datasets: misogyny, pornography, and malignant stereotypescited by 327
  17. 2019High Frequency Component Helps Explain the Generalization of Convolutional Neural Networkscited by 310
  18. 2021Unsolved Problems in ML Safetycited by 292
  19. 2019A Fourier Perspective on Model Robustness in Computer Visioncited by 288
  20. 2021MOS: Towards Scaling Out-of-distribution Detection for Large Semantic Spacecited by 233
  21. 2022Delving into Out-of-Distribution Detection with Vision-Language Representationscited by 216
  22. 2022Outlier Suppression: Pushing the Limit of Low-bit Transformer Language Modelscited by 181
  23. 2023OpenOOD v1.5: Enhanced Benchmark for Out-of-Distribution Detectioncited by 179
  24. 2019What Do Compressed Deep Neural Networks Forget?cited by 171
  25. 2020Characterising Bias in Compressed Modelscited by 167
  26. 2021The Low-Rank Simplicity Bias in Deep Networkscited by 154
  27. 2023CLIPN for Zero-Shot OOD Detection: Teaching CLIP to Say Nocited by 130
  28. 2022Q-ViT: Accurate and Fully Quantized Low-bit Vision Transformercited by 108
  29. 2023LoCoOp: Few-Shot Out-of-Distribution Detection via Prompt Learningcited by 107
  30. 2025A Sober Look at Progress in Language Model Reasoning: Pitfalls and Paths to Reproducibilitycited by 93
  31. 2023Towards Accurate Post-Training Quantization for Vision Transformercited by 71
  32. 2024Negative Label Guided OOD Detection with Pretrained Vision-Language Modelscited by 66
  33. 2022Full-Spectrum Out-of-Distribution Detectioncited by 63
  34. 2024A Closer Look at the Robustness of Contrastive Language-Image Pre-Training (CLIP)cited by 44
  35. 2024A Sober Look at the Robustness of CLIPs to Spurious Featurescited by 38
  36. 2024Generalized Out-of-Distribution Detection and Beyond in Vision Language Model Era: A Surveycited by 36
  37. 2023Does CLIP's Generalization Performance Mainly Stem from High Train-Test Similarity?cited by 35
  38. 2024Envisioning Outlier Exposure by Large Language Models for Out-of-Distribution Detectioncited by 32

50 most cited of 56 All 56 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026How Robustly do LLMs Understand Execution Semantics?cited by 2
  2. 2026Which Decisions Low-Bit Quantization Breaks, and How to Predict Themcited by 1
  3. 2026Edge Reliability Gap in Vision-Language Models: Quantifying Failure Modes of Compressed VLMs Under Visual Corruptioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025An Empirical Analysis of VLM-based OOD Detection: Mechanisms, Advantages, and Sensitivity
  2. 2024An Empirical Study Into What Matters for Calibrating Vision-Language Models
  3. 2026VLM-RobustBench: A Comprehensive Benchmark for Robustness of Vision-Language Models
  4. 2026Rethinking Practical and Efficient Quantization Calibration for Vision-Language Models
  5. 2025VLMInferSlow: Evaluating the Efficiency Robustness of Large Vision-Language Models as a Service
  6. 2025Bi-VLM: Pushing Ultra-Low Precision Post-Training Quantization Boundaries in Vision-Language Models
  7. 2026VLAQuantBench: Closed-Loop Evaluation of Post-Training Quantization for Vision-Language-Action Models
  8. 2026Rethinking Small VLM Quantization: From Component-Wise Analysis to Hardware-Aware Edge Deployment