mlsanityData to 14 Oct 2026

A Structured Framework for Evaluating and Enhancing Interpretive Capabilities of Multimodal LLMs in Culturally Situated Tasks

arXiv 2509.23208 ↗First posted 27 Sep 2025cs.CLcited by 4

On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning

Abstract (arXiv)

This study aims to test and evaluate the capabilities and characteristics of current mainstream Visual Language Models (VLMs) in generating critiques for traditional Chinese painting. To achieve this, we first developed a quantitative framework for Chinese painting critique. This framework was constructed by extracting multi-dimensional evaluative features covering evaluative stance, feature focus, and commentary quality from human expert critiques using a zero-shot classification model. Based on these features, several representative critic personas were defined and quantified. This framework was then employed to evaluate selected VLMs such as Llama, Qwen, or Gemini. The experimental design involved persona-guided prompting to assess the VLM's ability to generate critiques from diverse perspectives. Our findings reveal the current performance levels, strengths, and areas for improvement of VLMs in the domain of art critique, offering insights into their potential and limitations in complex semantic understanding and content generation tasks. The code used for our experiments can be publicly accessed at: https://github.com/yha9806/VULCA-EMNLP2025.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 14 of its 42 references that are in the map. First posted on arXiv 27 Sep 2025.

Builds on

14 of 42 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024GalleryGPT: Analyzing Paintings with Large Multimodal Models1 Aug
  2. 2024Benchmarking Vision Language Models for Cultural Understanding15 Jul
  3. 2024Creating a Lens of Chinese Culture: A Multimodal Dataset for Chinese Pun Rebus Art Understanding14 Jun
  4. 2023Ferret: Refer and Ground Anything Anywhere at Any Granularity11 Oct

Other references · 10

  1. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  2. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  3. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,682
  4. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,656
  5. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,141
  6. 2023SEED-Bench: Benchmarking Multimodal LLMs with Generative Comprehensioncited by 940
  7. 2023C-Pack: Packed Resources For General Chinese Embeddingscited by 578
  8. 2023PersonaLLM: Investigating the Ability of Large Language Models to Express Personality Traitscited by 234
  9. 2023Evaluating Large Language Models: A Comprehensive Surveycited by 202
  10. 2023ArtGPT-4: Towards Artistic-understanding Large Vision-Language Models with Enhanced Adaptercited by 14

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026Tears or Cheers? Benchmarking LLMs via Culturally Elicited Distinct Affective Responsescited by 6
  2. 2026VULCA-Bench: A Multicultural Vision-Language Benchmark for Evaluating Cultural Understandingcited by 3
  3. 2026Cross-Cultural Expert-Level Art Critique Evaluation with Vision-Language Modelscited by 1
  4. 2026HanMoVLM: Large Vision-Language Models for Professional Artistic Painting Evaluationcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2026Can MLLMs Critique Like Humans? Evaluating Open-Ended Aesthetic Reasoning in Multimodal Large Language Models
  2. 2026AesCanvas: A Large-Scale Dataset and Benchmark for Aesthetic Critique and Contextual Suitability
  3. 2024Can MLLMs Understand the Deep Implication Behind Chinese Images?
  4. 2026How LLMs See Creativity: Zero-Shot Scoring of Visual Creativity with Interpretable Reasoning
  5. 2026MUSE: Benchmarking Large Vision-Language Models on Multi-Modal Understanding in Situated Education
  6. 2026MuseBench: Benchmarking Intent-Level Audiovisual Arts Understanding in MLLMs
  7. 2026Aesthetic Assessment of Chinese Handwritings Based on Vision Language Models
  8. 2025VisTW: Benchmarking Vision-Language Models for Traditional Chinese in Taiwan