mlsanityData to 7 Oct 2026

Basic Category Usage in Vision Language Models

arXiv 2503.12530 ↗First posted 16 Mar 2025cs.CLcited by 0

On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning

Abstract (arXiv)

The field of psychology has long recognized a basic level of categorization that humans use when labeling visual stimuli, a term coined by Rosch in 1976. This level of categorization has been found to be used most frequently, to have higher information density, and to aid in visual language tasks with priming in humans. Here, we investigate basic-level categorization in two recently released, open-source vision-language models (VLMs). This paper demonstrates that Llama 3.2 Vision Instruct (11B) and Molmo 7B-D both prefer basic-level categorization consistent with human behavior. Moreover, the models' preferences are consistent with nuanced human behaviors like the biological versus non-biological basic level effects and the well-established expert basic level shift, further suggesting that VLMs acquire complex cognitive categorization behaviors from the human data on which they are trained. We also find our expert prompting methods demonstrate lower accuracy then our non-expert prompting methods, contradicting popular thought regarding the use of expertise prompting methods.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 8 of its 15 references that are in the map. First posted on arXiv 16 Mar 2025.

Builds on

8 of 15 references.

References · 8

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015VQA: Visual Question Answeringcited by 2,489
  3. 2023ExpertPrompting: Instructing Large Language Models to be Distinguished Expertscited by 140
  4. 2021Do language models learn typicality judgments from text?cited by 32
  5. 2024Multi-expert Prompting Improves Reliability, Safety, and Usefulness of Large Language Modelscited by 13
  6. 2023Using Artificial Populations to Study Psychological Phenomena in Neural Modelscited by 8
  7. 2024How Well Do Deep Learning Models Capture Human Concepts? The Case of the Typicality Effectcited by 7
  8. 2024Large Language Model Recall Uncertainty is Modulated by the Fan Effectcited by 7

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025The in-context inductive biases of vision-language models differ across modalities
  2. 2025Human Cognitive Benchmarks Reveal Foundational Visual Gaps in MLLMs
  3. 2025Seeing to Act, Prompting to Specify: A Bayesian Factorization of Vision Language Action Policy
  4. 2024Analyzing the Roles of Language and Vision in Learning from Limited Data
  5. 2025Generalizing vision-language models to novel domains: A comprehensive survey
  6. 2023Learning Hierarchical Prompt with Structured Linguistic Knowledge for Vision-Language Models
  7. 2025Not There Yet: Evaluating Vision Language Models in Simulating the Visual Perception of People with Low Vision
  8. 2025Visual Language Models show widespread visual deficits on neuropsychological tests