mlsanityData to 14 Oct 2026

Cross-Cultural Expert-Level Art Critique Evaluation with Vision-Language Models

arXiv 2601.07984 ↗First posted 12 Jan 2026cs.CLcited by 1

On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning

Abstract (arXiv)

Vision-Language Models (VLMs) excel at visual description yet remain under-validated for cultural interpretation. Existing benchmarks assess perception without interpretation, and common evaluation proxies, such as automated metrics and LLM-judge averaging, are unreliable for culturally sensitive generative tasks. We address this measurement gap with a tri-tier evaluation framework grounded in art-theoretical constructs (Section 2). The framework operationalises cultural understanding through five levels (L1--L5) and 165 culture-specific dimensions across six traditions: Tier I computes automated quality indicators, Tier II applies rubric-based single-judge scoring, and Tier III calibrates the aggregate score to human expert ratings via sigmoid calibration. Applied to 15 VLMs across 294 evaluation pairs, the validated instrument reveals that (i) automated metrics and judge scoring measure different constructs, establishing single-judge calibration as the more reliable alternative; (ii) cultural understanding degrades from visual description (L1--L2) to cultural interpretation (L3--L5); and (iii) Western art samples consistently receive higher scores than non-Western ones. To our knowledge, this is the first cross-cultural evaluation instrument for generative art critique, providing a reproducible methodology for auditing VLM cultural competence. Framework code is available at https://github.com/yha9806/VULCA-Framework.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 18 of its 52 references that are in the map. First posted on arXiv 12 Jan 2026.

Builds on

18 of 52 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025Seeing Symbols, Missing Cultures: Probing Vision-Language Models' Reasoning on Fire Imagery and Cultural Meaning27 Sep
  2. 2025A Structured Framework for Evaluating and Enhancing Interpretive Capabilities of Multimodal LLMs in Culturally Situated Tasks27 Sep
  3. 2025GIMMICK -- Globally Inclusive Multimodal Multitask Cultural Knowledge Benchmarking19 Feb
  4. 2024Have Large Vision-Language Models Mastered Art History?5 Sep
  5. 2024Benchmarking Vision Language Models for Cultural Understanding15 Jul
  6. 2024Prometheus-Vision: Vision-Language Model as a Judge for Fine-Grained Evaluation12 Jan

Other references · 12

  1. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  2. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,516
  3. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,276
  4. 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,239
  5. 2022Winoground: Probing Vision and Language Models for Visio-Linguistic Compositionalitycited by 529
  6. 2015Large-scale Classification of Fine-Art Paintings: Learning The Right Metric on The Right Featurecited by 182
  7. 2021ArtEmis: Affective Language for Visual Artcited by 104
  8. 2026Grading Scale Impact on LLM-as-a-Judge: Human-LLM Alignment Is Highest on 0-5 Grading Scalecited by 20
  9. 2017OmniArt: Multi-task Deep Learning for Artistic Data Analysiscited by 16
  10. 2024Evaluating Fairness in Large Vision-Language Models Across Diverse Demographic Attributes and Promptscited by 15
  11. 2025Towards Cross-modal Retrieval in Chinese Cultural Heritage Documents: Dataset and Solutioncited by 4
  12. 2023Computational Approaches for Traditional Chinese Painting: From the "Six Principles of Painting" Perspectivecited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026HanMoVLM: Large Vision-Language Models for Professional Artistic Painting Evaluationcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2026VULCA-Bench: A Multicultural Vision-Language Benchmark for Evaluating Cultural Understanding
  2. 2025Evaluation of Cultural Competence of Vision-Language Models
  3. 2025Exposing Blindspots: Cultural Bias Evaluation in Generative Image Models
  4. 2026MUSE: Benchmarking Large Vision-Language Models on Multi-Modal Understanding in Situated Education
  5. 2026Can Vision Language Models Assess Graphic Design Aesthetics? A Benchmark, Evaluation, and Dataset Perspective
  6. 2026Visual Aesthetic Benchmark: Can Frontier Models Judge Beauty?
  7. 2026Fine-Tuning a Large Vision-Language Model for Artwork's Scoring and Critique
  8. 2026Disentangling Algorithmic Bias from Archival Artifacts: A Controlled Audit of Vision-Language Model Valuation in Metropolitan Museum Archives