# A Structured Framework for Evaluating and Enhancing Interpretive Capabilities of Multimodal LLMs in Culturally Situated Tasks

arXiv: https://arxiv.org/abs/2509.23208 · HTML: https://mlsanity.com/paper/2509.23208/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 14 of its 42 references that are in the map.
- First posted on arXiv 27 Sep 2025.

## Placed in

- Multimodal LLMs / vision-language models / [VLM benchmarks and visual reasoning](https://mlsanity.com/mllm/vlm-benchmarks-and-visual-reasoning/)

## Builds on (most cited first, 14 of 14)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference](https://mlsanity.com/paper/1704.05426/) (2017)
- [Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations](https://mlsanity.com/paper/1602.07332/) (2016)
- [MMBench: Is Your Multi-modal Model an All-around Player?](https://mlsanity.com/paper/2307.06281/) (2023)
- [SEED-Bench: Benchmarking Multimodal LLMs with Generative Comprehension](https://mlsanity.com/paper/2307.16125/) (2023)
- [C-Pack: Packed Resources For General Chinese Embeddings](https://mlsanity.com/paper/2309.07597/) (2023)
- [Ferret: Refer and Ground Anything Anywhere at Any Granularity](https://mlsanity.com/paper/2310.07704/) (2023)
- [PersonaLLM: Investigating the Ability of Large Language Models to Express Personality Traits](https://mlsanity.com/paper/2305.02547/) (2023)
- [Evaluating Large Language Models: A Comprehensive Survey](https://mlsanity.com/paper/2310.19736/) (2023)
- [Benchmarking Vision Language Models for Cultural Understanding](https://mlsanity.com/paper/2407.10920/) (2024)
- [GalleryGPT: Analyzing Paintings with Large Multimodal Models](https://mlsanity.com/paper/2408.00491/) (2024)
- [ArtGPT-4: Towards Artistic-understanding Large Vision-Language Models with Enhanced Adapter](https://mlsanity.com/paper/2305.07490/) (2023)
- [Creating a Lens of Chinese Culture: A Multimodal Dataset for Chinese Pun Rebus Art Understanding](https://mlsanity.com/paper/2406.10318/) (2024)

## Built on by (most cited first, 4 of 4)

- [Tears or Cheers? Benchmarking LLMs via Culturally Elicited Distinct Affective Responses](https://mlsanity.com/paper/2601.13024/) (2026)
- [VULCA-Bench: A Multicultural Vision-Language Benchmark for Evaluating Cultural Understanding](https://mlsanity.com/paper/2601.07986/) (2026)
- [Cross-Cultural Expert-Level Art Critique Evaluation with Vision-Language Models](https://mlsanity.com/paper/2601.07984/) (2026)
- [HanMoVLM: Large Vision-Language Models for Professional Artistic Painting Evaluation](https://mlsanity.com/paper/2603.10814/) (2026)

## Abstract (arXiv)

This study aims to test and evaluate the capabilities and characteristics of current mainstream Visual Language Models (VLMs) in generating critiques for traditional Chinese painting. To achieve this, we first developed a quantitative framework for Chinese painting critique. This framework was constructed by extracting multi-dimensional evaluative features covering evaluative stance, feature focus, and commentary quality from human expert critiques using a zero-shot classification model. Based on these features, several representative critic personas were defined and quantified. This framework was then employed to evaluate selected VLMs such as Llama, Qwen, or Gemini. The experimental design involved persona-guided prompting to assess the VLM's ability to generate critiques from diverse perspectives. Our findings reveal the current performance levels, strengths, and areas for improvement of VLMs in the domain of art critique, offering insights into their potential and limitations in complex semantic understanding and content generation tasks. The code used for our experiments can be publicly accessed at: https://github.com/yha9806/VULCA-EMNLP2025.