mlsanityData to 14 Oct 2026

CArtBench: Evaluating Vision-Language Models on Chinese Art Understanding, Interpretation, and Authenticity

arXiv 2604.11632 ↗First posted 13 Apr 2026cs.CLcited by 1

On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning

Abstract (arXiv)

We introduce CARTBENCH, a museum-grounded benchmark for evaluating vision-language models (VLMs) on Chinese artworks beyond short-form recognition and QA. CARTBENCH comprises four subtasks: CURATORQA for evidence-grounded recognition and reasoning, CATALOGCAPTION for structured four-section expert-style appreciation, REINTERPRET for defensible reinterpretation with expert ratings, and CONNOISSEURPAIRS for diagnostic authenticity discrimination under visually similar confounds. CARTBENCH is built by aligning image-bearing Palace Museum objects from Wikidata with authoritative catalog pages, spanning five art categories across multiple dynasties. Across nine representative VLMs, we find that high overall CURATORQA accuracy can mask sharp drops on hard evidence linking and style-to-period inference; long-form appreciation remains far from expert references; and authenticity-oriented diagnostic discrimination stays near chance, underscoring the difficulty of connoisseur-level reasoning for current models.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 14 of its 32 references that are in the map. First posted on arXiv 13 Apr 2026.

Builds on

14 of 32 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024Towards Cross-Lingual Explanation of Artwork in Large-scale Vision Language Models3 Sep
  2. 2024Benchmarking Vision Language Models for Cultural Understanding15 Jul
  3. 2024See It from My Perspective: How Language Affects Cultural Bias in Image Understanding17 Jun

Other references · 11

  1. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  2. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  3. 2015VQA: Visual Question Answeringcited by 2,506
  4. 2014CIDEr: Consensus-based Image Description Evaluationcited by 2,046
  5. 2020Shortcut Learning in Deep Neural Networkscited by 1,969
  6. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,652
  7. 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,239
  8. 2017Don't Just Assume; Look and Answer: Overcoming Priors for Visual Question Answeringcited by 339
  9. 2021ArtEmis: Affective Language for Visual Artcited by 104
  10. 2024CMMU: A Benchmark for Chinese Multi-modal Multi-type Question Understanding and Reasoningcited by 31
  11. 2024No Culture Left Behind: ArtELingo-28, a Benchmark of WikiArt with Captions in 28 Languagescited by 19

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026CulMind: Benchmarking Multimodal Understanding and Reasoning in Chinese Cultural Heritagecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026HanMoVLM: Large Vision-Language Models for Professional Artistic Painting Evaluation
  2. 2024Can MLLMs Understand the Deep Implication Behind Chinese Images?
  3. 2024Have Large Vision-Language Models Mastered Art History?
  4. 2024Creating a Lens of Chinese Culture: A Multimodal Dataset for Chinese Pun Rebus Art Understanding
  5. 2026VULCA-Bench: A Multicultural Vision-Language Benchmark for Evaluating Cultural Understanding
  6. 2025Benchmarking Vision-Language Models on Chinese Ancient Documents: From OCR to Knowledge Reasoning
  7. 2026Cross-Cultural Expert-Level Art Critique Evaluation with Vision-Language Models
  8. 2026Disentangling Algorithmic Bias from Archival Artifacts: A Controlled Audit of Vision-Language Model Valuation in Metropolitan Museum Archives