mlsanityData to 7 Oct 2026

Appear2Meaning: A Cross-Cultural Benchmark for Structured Cultural Metadata Inference from Images

arXiv 2604.07338 ↗First posted 8 Apr 2026cs.CV cs.CL cs.MMcited by 0

On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning

Abstract (arXiv)

Recent advances in vision-language models (VLMs) have improved image captioning for cultural heritage. However, inferring structured cultural metadata (e.g., creator, origin, period) from visual input remains underexplored. We introduce a multi-category, cross-cultural benchmark for this task and evaluate VLMs using an LLM-as-Judge framework that measures semantic alignment with reference annotations. To assess cultural reasoning, we report exact-match, partial-match, and attribute-level accuracy across cultural regions. Results show that models capture fragmented signals and exhibit substantial performance variation across cultures and metadata types, leading to inconsistent and weakly grounded predictions. These findings highlight the limitations of current VLMs in structured cultural metadata inference beyond visual perception.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 18 of its 50 references that are in the map. First posted on arXiv 8 Apr 2026.

Builds on

18 of 50 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025Describe Anything: Detailed Localized Image and Video Captioning22 Apr
  2. 2025CultureVLM: Characterizing and Improving Cultural Understanding of Vision-Language Models for over 100 Countries2 Jan
  3. 2024The Power of Many: Multi-Agent Multimodal Models for Cultural Image Captioning18 Nov
  4. 2024KALE: An Artwork Image Captioning System Augmented with Heterogeneous Graph17 Sep

Other references · 14

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
  3. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,219
  4. 2015Show, Attend and Tell: Neural Image Caption Generation with Visual Attentioncited by 1,421
  5. 2017Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answeringcited by 1,310
  6. 2020Oscar: Object-Semantics Aligned Pre-training for Vision-Language Taskscited by 1,213
  7. 2014Show and Tell: A Neural Image Caption Generatorcited by 877
  8. 2022PaLI: A Jointly-Scaled Multilingual Language-Image Modelcited by 762
  9. 2022OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Frameworkcited by 742
  10. 2023Diffusion Based Augmentation for Captioning and Retrieval in Cultural Heritagecited by 12
  11. 2025EmoArt: A Multidimensional Dataset for Emotion-Aware Artistic Generationcited by 10
  12. 2024CompCap: Improving Multimodal Large Language Models with Composite Captionscited by 8
2 more
  1. 2025ArtSeek: Deep artwork understanding via multimodal in-context reasoning and late interaction retrievalcited by 5
  2. 2025DenseAnnotate: Enabling Scalable Dense Caption Collection for Images and 3D Scenes via Spoken Descriptionscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Seeing Culture: A Benchmark for Visual Reasoning and Grounding
  2. 2025RAVENEA: A Benchmark for Multimodal Retrieval-Augmented Visual Culture Understanding
  3. 2025CultureCLIP: Empowering CLIP with Cultural Awareness through Synthetic Images and Contextualized Captions
  4. 2025Hanfu-Bench: A Multimodal Benchmark on Cross-Temporal Cultural Understanding and Transcreation
  5. 2025CuRe: Cultural Gaps in the Long Tail of Text-to-Image Systems
  6. 2026ChinaHeritaQA: A Culturally-Grounded Visual Question Answering Dataset for World Heritage Sites in China
  7. 2024CIC: A Framework for Culturally-Aware Image Captioning
  8. 2024Crossroads of Continents: Automated Artifact Extraction for Cultural Adaptation with Large Multimodal Models