mlsanityData to 14 Oct 2026

BLEnD-Vis: Benchmarking Multimodal Cultural Understanding in Vision Language Models

arXiv 2510.11178 ↗First posted 13 Oct 2025cs.CV cs.CYcited by 3

On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning

Abstract (arXiv)

As vision-language models (VLMs) are deployed globally, their ability to understand culturally situated knowledge becomes essential. Yet, existing evaluations largely assess static recall or isolated visual grounding, leaving unanswered whether VLMs possess robust and transferable cultural understanding. We introduce BLEnD-Vis, a multimodal, multicultural benchmark designed to evaluate the robustness of everyday cultural knowledge in VLMs across linguistic rephrasings and visual modalities. Building on the BLEnD dataset, BLEnD-Vis constructs 313 culturally grounded question templates spanning 16 regions and generates three aligned multiple-choice formats: (i) a text-only baseline querying from Region rightarrow Entity, (ii) an inverted text-only variant (Entity rightarrow Region), and (iii) a VQA-style version of (ii) with generated images. The resulting benchmark comprises 4,916 images and over 21,000 multiple-choice questions (MCQ) instances, validated through human annotation. BLEnD-Vis reveals significant fragility in current VLM cultural knowledge; models exhibit performance drops under linguistic rephrasing. While visual cues often aid performance, low cross-modal consistency highlights the challenges of robustly integrating textual and visual understanding, particularly in lower-resource regions. BLEnD-Vis thus provides a crucial testbed for systematically analysing cultural robustness and multimodal grounding, exposing limitations and guiding the development of more culturally competent VLMs. Code is available at https://github.com/Social-AI-Studio/BLEnD-Vis.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 17 of its 39 references that are in the map. First posted on arXiv 13 Oct 2025.

Builds on

17 of 39 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025Seeing Culture: A Benchmark for Visual Reasoning and Grounding20 Sep
  2. 2025RAVENEA: A Benchmark for Multimodal Retrieval-Augmented Visual Culture Understanding20 May
  3. 2024NVILA: Efficient Frontier Visual Language Models5 Dec
  4. 2024PaliGemma 2: A Family of Versatile VLMs for Transfer4 Dec
  5. 2024Benchmarking Vision Language Models for Cultural Understanding15 Jul

Other references · 12

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  2. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
  3. 2024BLEnD: A Benchmark for LLMs on Everyday Knowledge in Diverse Cultures and Languagescited by 173
  4. 2024Towards Measuring and Modeling "Culture" in LLMs: A Surveycited by 167
  5. 2024Investigating Cultural Alignment of Large Language Modelscited by 143
  6. 2024Survey of Cultural Awareness in Language Models: Text and Beyondcited by 103
  7. 2023SeaEval for Multilingual Foundation Models: From Cross-Lingual Alignment to Cultural Reasoningcited by 94
  8. 2024Evaluating Cultural and Social Awareness of LLM Web Agentscited by 45
  9. 2024CulturalBench: A Robust, Diverse, and Challenging Cultural Benchmark by Human-AI CulturalTeamingcited by 30
  10. 2024B-AVIBench: Towards Evaluating the Robustness of Large Vision-Language Model on Black-box Adversarial Visual-Instructionscited by 26
  11. 2025Unmasking Implicit Bias: Evaluating Persona-Prompted LLM Responses in Power-Disparate Social Scenarioscited by 26
  12. 2024CROPE: Evaluating In-Context Adaptation of Vision and Language Models to Culture-Specific Conceptscited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026EgoGroups: A Benchmark For Detecting Social Groups of People in the Wildcited by 0
  2. 2026Not Truly Multilingual: Script Consistency as a Missing Dimension in VLM Evaluationcited by 0
  3. 2026PUMA: A Polish Benchmark for Culturally Grounded Multimodal Understandingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025CultureVLM: Characterizing and Improving Cultural Understanding of Vision-Language Models for over 100 Countries
  2. 2025Rice-VL: Evaluating Vision-Language Models for Cultural Understanding Across ASEAN Countries
  3. 2024Evaluating Visual and Cultural Interpretation: The K-Viscuit Benchmark with Human-VLM Collaboration
  4. 2025Grounding Multilingual Multimodal LLMs With Cultural Knowledge
  5. 2024From Local Concepts to Universals: Evaluating the Multicultural Understanding of Vision-Language Models
  6. 2025World in a Frame: Understanding Culture Mixing as a New Challenge for Vision-Language Models
  7. 2026VULCA-Bench: A Multicultural Vision-Language Benchmark for Evaluating Cultural Understanding
  8. 2025IndicVisionBench: Benchmarking Cultural and Multilingual Understanding in VLMs