mlsanityData to 7 Oct 2026

ImageNetVC: Zero- and Few-Shot Visual Commonsense Evaluation on 1000 ImageNet Categories

arXiv 2305.15028 ↗First posted 24 May 2023cs.CL cs.AI cs.CVcited by 4

Not on the map. Why?

Abstract (arXiv)

Recently, Large Language Models (LLMs) have been serving as general-purpose interfaces, posing a significant demand for comprehensive visual knowledge. However, it remains unclear how well current LLMs and their visually augmented counterparts (VaLMs) can master visual commonsense knowledge. To investigate this, we propose ImageNetVC, a human-annotated dataset specifically designed for zero- and few-shot visual commonsense evaluation across 1,000 ImageNet categories. Utilizing ImageNetVC, we benchmark the fundamental visual commonsense knowledge of both unimodal LLMs and VaLMs. Furthermore, we analyze the factors affecting the visual commonsense knowledge of large-scale models, providing insights into the development of language models enriched with visual commonsense knowledge. Our code and dataset are available at https://github.com/hemingkx/ImageNetVC.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 26 of its 38 references that are in the map. First posted on arXiv 24 May 2023.

Builds on

26 of 38 references.

References · 26

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  5. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  6. 2017On Calibration of Modern Neural Networkscited by 4,860
  7. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  8. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  9. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,498
  10. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,346
  11. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,793
  12. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,734
14 more
  1. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,371
  2. 2020Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inferencecited by 934
  3. 2022GPT-NeoX-20B: An Open-Source Autoregressive Language Modelcited by 711
  4. 2021Multimodal Few-Shot Learning with Frozen Language Modelscited by 657
  5. 2022LM-Nav: Robotic Navigation with Large Pre-Trained Models of Language, Vision, and Actioncited by 435
  6. 2022Language Models are General-Purpose Interfacescited by 96
  7. 2021MAGMA -- Multimodal Augmentation of Generative Models through Adapter-based Finetuningcited by 85
  8. 2022Things not Written in Text: Exploring Spatial Commonsense from Visual Signalscited by 41
  9. 2022Visual Commonsense in Pretrained Unimodal and Multimodal Modelscited by 39
  10. 2016Are Elephants Bigger than Butterflies? Reasoning about Sizes of Objectscited by 22
  11. 2023Can Language Models Understand Physical Concepts?cited by 19
  12. 2021Transferring Knowledge from Vision to Language: How to Achieve it and how to Measure it?cited by 18
  13. 2022Z-LaVI: Zero-Shot Language Solver Fueled by Visual Imaginationcited by 18
  14. 2021Premise-based Multimodal Reasoning: Conditional Inference on Joint Textual and Visual Cluescited by 8

Built on by

Cited by year

Citing papers per year: 2024: 3; 2025: 1.

4 papers.

  1. 2024Can Large Multimodal Models Uncover Deep Semantics Behind Images?cited by 16
  2. 2024VCD: A Dataset for Visual Commonsense Discovery in Imagescited by 5
  3. 2025Beyond Single Frames: Can LMMs Comprehend Temporal and Contextual Narratives in Image Sequences?cited by 4
  4. 2024LaMI: Augmenting Large Language Models via Late Multi-Image Fusioncited by 1