VULCA-Bench: A Multicultural Vision-Language Benchmark for Evaluating Cultural Understanding
On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning
Abstract (arXiv)
We introduce VULCA-Bench, a multicultural art-critique benchmark for evaluating Vision-Language Models' (VLMs) cultural understanding beyond surface-level visual perception. Existing VLM benchmarks predominantly measure L1-L2 capabilities (object recognition, scene description, and factual question answering) while under-evaluate higher-order cultural interpretation. VULCA-Bench contains 7,410 matched image-critique pairs spanning eight cultural traditions, with Chinese-English bilingual coverage. We operationalise cultural understanding using a five-layer framework (L1-L5, from Visual Perception to Philosophical Aesthetics), instantiated as 225 culture-specific dimensions and supported by expert-written bilingual critiques. Our pilot results indicate that higher-layer reasoning (L3-L5) is consistently more challenging than visual and technical analysis (L1-L2). The dataset, evaluation scripts, and annotation tools are available under CC BY 4.0 at https://github.com/yha9806/VULCA-Bench.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 15 of its 28 references that are in the map. First posted on arXiv 12 Jan 2026.
Builds on
15 of 28 references.
Closest ancestors · 7
Same direction, newest first
- 2025Seeing Symbols, Missing Cultures: Probing Vision-Language Models' Reasoning on Fire Imagery and Cultural Meaning27 Sep
- 2025A Structured Framework for Evaluating and Enhancing Interpretive Capabilities of Multimodal LLMs in Culturally Situated Tasks27 Sep
- 2025GIMMICK -- Globally Inclusive Multimodal Multitask Cultural Knowledge Benchmarking19 Feb
- 2025CultureVLM: Characterizing and Improving Cultural Understanding of Vision-Language Models for over 100 Countries2 Jan
- 2024Can MLLMs Understand the Deep Implication Behind Chinese Images?17 Oct
- 2024Benchmarking Vision Language Models for Cultural Understanding15 Jul
- 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts3 Oct
Other references · 8
- 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,498
- 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,449
- 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,288
- 2015Large-scale Classification of Fine-Art Paintings: Learning The Right Metric on The Right Featurecited by 181
- 2021ArtEmis: Affective Language for Visual Artcited by 103
- 2024Massively Multi-Cultural Knowledge Acquisition & LM Benchmarkingcited by 63
- 2017OmniArt: Multi-task Deep Learning for Artistic Data Analysiscited by 16
- 2025Towards Cross-modal Retrieval in Chinese Cultural Heritage Documents: Dataset and Solutioncited by 4
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2025Seeing Culture: A Benchmark for Visual Reasoning and Grounding
- 2025BLEnD-Vis: Benchmarking Multimodal Cultural Understanding in Vision Language Models
- 2024CVLUE: A New Benchmark Dataset for Chinese Vision-Language Understanding Evaluation
- 2024From Local Concepts to Universals: Evaluating the Multicultural Understanding of Vision-Language Models
- 2026Cross-Cultural Expert-Level Art Critique Evaluation with Vision-Language Models
- 2025Rice-VL: Evaluating Vision-Language Models for Cultural Understanding Across ASEAN Countries
- 2025Evaluation of Cultural Competence of Vision-Language Models
- 2024Evaluating Visual and Cultural Interpretation: The K-Viscuit Benchmark with Human-VLM Collaboration