# Cross-Cultural Expert-Level Art Critique Evaluation with Vision-Language Models

arXiv: https://arxiv.org/abs/2601.07984 · HTML: https://mlsanity.com/paper/2601.07984/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 18 of its 52 references that are in the map.
- First posted on arXiv 12 Jan 2026.

## Placed in

- Multimodal LLMs / vision-language models / [VLM benchmarks and visual reasoning](https://mlsanity.com/mllm/vlm-benchmarks-and-visual-reasoning/)

## Builds on (most cited first, 18 of 18)

- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering](https://mlsanity.com/paper/1612.00837/) (2016)
- [G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment](https://mlsanity.com/paper/2303.16634/) (2023)
- [MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilities](https://mlsanity.com/paper/2308.02490/) (2023)
- [Winoground: Probing Vision and Language Models for Visio-Linguistic Compositionality](https://mlsanity.com/paper/2204.03162/) (2022)
- [Large-scale Classification of Fine-Art Paintings: Learning The Right Metric on The Right Feature](https://mlsanity.com/paper/1505.00855/) (2015)
- [Prometheus-Vision: Vision-Language Model as a Judge for Fine-Grained Evaluation](https://mlsanity.com/paper/2401.06591/) (2024)
- [ArtEmis: Affective Language for Visual Art](https://mlsanity.com/paper/2101.07396/) (2021)
- [Benchmarking Vision Language Models for Cultural Understanding](https://mlsanity.com/paper/2407.10920/) (2024)
- [Grading Scale Impact on LLM-as-a-Judge: Human-LLM Alignment Is Highest on 0-5 Grading Scale](https://mlsanity.com/paper/2601.03444/) (2026)
- [OmniArt: Multi-task Deep Learning for Artistic Data Analysis](https://mlsanity.com/paper/1708.00684/) (2017)
- [GIMMICK -- Globally Inclusive Multimodal Multitask Cultural Knowledge Benchmarking](https://mlsanity.com/paper/2502.13766/) (2025)
- [Evaluating Fairness in Large Vision-Language Models Across Diverse Demographic Attributes and Prompts](https://mlsanity.com/paper/2406.17974/) (2024)
- [Have Large Vision-Language Models Mastered Art History?](https://mlsanity.com/paper/2409.03521/) (2024)
- [Towards Cross-modal Retrieval in Chinese Cultural Heritage Documents: Dataset and Solution](https://mlsanity.com/paper/2505.10921/) (2025)
- [A Structured Framework for Evaluating and Enhancing Interpretive Capabilities of Multimodal LLMs in Culturally Situated Tasks](https://mlsanity.com/paper/2509.23208/) (2025)
- [Computational Approaches for Traditional Chinese Painting: From the "Six Principles of Painting" Perspective](https://mlsanity.com/paper/2307.14227/) (2023)
- [Seeing Symbols, Missing Cultures: Probing Vision-Language Models' Reasoning on Fire Imagery and Cultural Meaning](https://mlsanity.com/paper/2509.23311/) (2025)

## Built on by (most cited first, 1 of 1)

- [HanMoVLM: Large Vision-Language Models for Professional Artistic Painting Evaluation](https://mlsanity.com/paper/2603.10814/) (2026)

## Abstract (arXiv)

Vision-Language Models (VLMs) excel at visual description yet remain under-validated for cultural interpretation. Existing benchmarks assess perception without interpretation, and common evaluation proxies, such as automated metrics and LLM-judge averaging, are unreliable for culturally sensitive generative tasks. We address this measurement gap with a tri-tier evaluation framework grounded in art-theoretical constructs (Section 2). The framework operationalises cultural understanding through five levels (L1--L5) and 165 culture-specific dimensions across six traditions: Tier I computes automated quality indicators, Tier II applies rubric-based single-judge scoring, and Tier III calibrates the aggregate score to human expert ratings via sigmoid calibration. Applied to 15 VLMs across 294 evaluation pairs, the validated instrument reveals that (i) automated metrics and judge scoring measure different constructs, establishing single-judge calibration as the more reliable alternative; (ii) cultural understanding degrades from visual description (L1--L2) to cultural interpretation (L3--L5); and (iii) Western art samples consistently receive higher scores than non-Western ones. To our knowledge, this is the first cross-cultural evaluation instrument for generative art critique, providing a reproducible methodology for auditing VLM cultural competence. Framework code is available at https://github.com/yha9806/VULCA-Framework.