mlsanityData to 7 Oct 2026

Do Vision-Language Foundational models show Robust Visual Perception?

arXiv 2408.06781 ↗First posted 13 Aug 2024cs.CVcited by 0

On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning

Abstract (arXiv)

Recent advances in vision-language foundational models have enabled development of systems that can perform visual understanding and reasoning tasks. However, it is unclear if these models are robust to distribution shifts, and how their performance and generalization capabilities vary under changes in data distribution. In this project we strive to answer the question "Are vision-language foundational models robust to distribution shifts like human perception?" Specifically, we consider a diverse range of vision-language models and compare how the performance of these systems is affected by corruption based distribution shifts (such as textitmotion blur, fog, snow, gaussian noise) commonly found in practical real-world scenarios. We analyse the generalization capabilities qualitatively and quantitatively on zero-shot image classification task under aforementioned distribution shifts. Our code will be avaible at urlhttps://github.com/shivam-chandhok/CPSC-540-Project

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 9 of its 10 references that are in the map. First posted on arXiv 13 Aug 2024.

Builds on

9 of 10 references.

References · 9

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2023Visual Instruction Tuningcited by 7,792
  5. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  6. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,857
  7. 2017Deeper, Broader and Artier Domain Generalizationcited by 983
  8. 2021Intriguing Properties of Vision Transformerscited by 446
  9. 2021Delving Deep into the Generalization of Vision Transformers under Distribution Shiftscited by 85

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Analysing the Robustness of Vision-Language-Models to Common Corruptions
  2. 2026Step-Level Visual Grounding Faithfulness Predicts Out-of-Distribution Generalization in Long-Horizon Vision-Language Models
  3. 2024Are foundation models for computer vision good conformal predictors?
  4. 2026VLM-RobustBench: A Comprehensive Benchmark for Robustness of Vision-Language Models
  5. 2025Unveiling the Lack of LVLM Robustness to Fundamental Visual Variations: Why and Path Forward
  6. 2025Uniformity First: Uniformity-aware Test-time Adaptation of Vision-language Models against Image Corruption
  7. 2025On the Domain Robustness of Contrastive Vision-Language Models
  8. 2023Robust Fine-Tuning of Vision-Language Models for Domain Generalization