mlsanityData to 7 Oct 2026

Jako Tako or Fluent? Presenting PoVisLE: A Polish Vision-Language Evaluation

arXiv 2608.07763 ↗First posted 7 Aug 2026cs.CL cs.CVcited by 1

On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning

Abstract (arXiv)

Vision-language models (VLMs) have achieved strong performance on tasks such as image captioning, visual question answering, and image-to-text generation. However, they are predominantly trained on English-centric data, which limits their ability to handle culturally grounded visual understanding and leads to failures in interpreting region-specific meanings, symbolic content, and context-dependent visual cues. Existing benchmarks for cultural competence are often template-driven and focused on surface-level recognition, making them insufficient for evaluating deeper linguistic and pragmatic understanding in culturally situated settings. We introduce PoVisLE, a monocultural vision-language benchmark for Polish designed to evaluate culturally grounded multimodal understanding under a grounded evaluation paradigm, where language is interpreted in interaction with visual context. The dataset contains 1,117 images and 2,366 manually annotated VQA pairs. Overall, our dataset provides a controlled and challenging resource for assessing culturally grounded vision-language understanding beyond surface-level recognition.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 20 of its 28 references that are in the map. First posted on arXiv 7 Aug 2026.

Builds on

20 of 28 references.

Closest ancestors · 7

Same direction, newest first

  1. 2026Annotation-Efficient Vision-Language Model Adaptation to the Polish Language Using the LLaVA Framework15 Feb
  2. 2025Can VLMs Recall Factual Associations From Visual References?22 Aug
  3. 2025JEEM: Vision-Language Understanding in Four Arabic Dialects27 Mar
  4. 2024WorldCuisines: A Massive-Scale Benchmark for Multilingual and Multicultural Visual Question Answering on Global Cuisines16 Oct
  5. 2024Benchmarking Vision Language Models for Cultural Understanding15 Jul
  6. 2024CVLUE: A New Benchmark Dataset for Chinese Vision-Language Understanding Evaluation1 Jul
  7. 2024CVQA: Culturally-diverse Multilingual Visual Question Answering Benchmark10 Jun

Other references · 13

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  3. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,498
  4. 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,445
  5. 2019OK-VQA: A Visual Question Answering Benchmark Requiring External Knowledgecited by 1,114
  6. 2023Large Language Models Are Not Robust Multiple Choice Selectorscited by 454
  7. 2024Artifacts or Abduction: How Do LLMs Answer Multiple-Choice Questions Without the Question?cited by 68
  8. 2024Understanding the Capabilities and Limitations of Large Language Models for Cultural Commonsensecited by 50
  9. 2023StyloMetrix: An Open-Source Multilingual Tool for Representing Stylometric Vectorscited by 14
  10. 2025DRISHTIKON: A Multimodal Multilingual Benchmark for Testing Language Models' Understanding on Indian Culturecited by 13
  11. 2025LLMzSzL: a comprehensive LLM benchmark for Polishcited by 7
  12. 2025CUS-QA: Local-Knowledge-Oriented Open-Ended Question Answering Datasetcited by 4
1 more
  1. 2025Charting the European LLM Benchmarking Landscape: A New Taxonomy and a Set of Best Practicescited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Polish Medical Visual Question Answering: Vision-Language Models Underutilize Visual Evidencecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024From Local Concepts to Universals: Evaluating the Multicultural Understanding of Vision-Language Models
  2. 2025CultureVLM: Characterizing and Improving Cultural Understanding of Vision-Language Models for over 100 Countries
  3. 2025BLEnD-Vis: Benchmarking Multimodal Cultural Understanding in Vision Language Models
  4. 2024Evaluating Visual and Cultural Interpretation: The K-Viscuit Benchmark with Human-VLM Collaboration
  5. 2025IndicVisionBench: Benchmarking Cultural and Multilingual Understanding in VLMs
  6. 2025VLURes: Benchmarking VLM Visual and Linguistic Understanding in Low-Resource Languages
  7. 2025Kaleidoscope: In-language Exams for Massively Multilingual Vision Evaluation
  8. 2025Rice-VL: Evaluating Vision-Language Models for Cultural Understanding Across ASEAN Countries