# Appear2Meaning: A Cross-Cultural Benchmark for Structured Cultural Metadata Inference from Images

arXiv: https://arxiv.org/abs/2604.07338 · HTML: https://mlsanity.com/paper/2604.07338/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 18 of its 50 references that are in the map.
- First posted on arXiv 8 Apr 2026.

## Placed in

- Multimodal LLMs / vision-language models / [VLM benchmarks and visual reasoning](https://mlsanity.com/mllm/vlm-benchmarks-and-visual-reasoning/)

## Builds on (most cited first, 18 of 18)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning](https://mlsanity.com/paper/2305.06500/) (2023)
- [ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks](https://mlsanity.com/paper/1908.02265/) (2019)
- [Show, Attend and Tell: Neural Image Caption Generation with Visual Attention](https://mlsanity.com/paper/1502.03044/) (2015)
- [Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering](https://mlsanity.com/paper/1707.07998/) (2017)
- [Oscar: Object-Semantics Aligned Pre-training for Vision-Language Tasks](https://mlsanity.com/paper/2004.06165/) (2020)
- [Show and Tell: A Neural Image Caption Generator](https://mlsanity.com/paper/1411.4555/) (2014)
- [PaLI: A Jointly-Scaled Multilingual Language-Image Model](https://mlsanity.com/paper/2209.06794/) (2022)
- [OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Framework](https://mlsanity.com/paper/2202.03052/) (2022)
- [Describe Anything: Detailed Localized Image and Video Captioning](https://mlsanity.com/paper/2504.16072/) (2025)
- [CultureVLM: Characterizing and Improving Cultural Understanding of Vision-Language Models for over 100 Countries](https://mlsanity.com/paper/2501.01282/) (2025)
- [Diffusion Based Augmentation for Captioning and Retrieval in Cultural Heritage](https://mlsanity.com/paper/2308.07151/) (2023)
- [EmoArt: A Multidimensional Dataset for Emotion-Aware Artistic Generation](https://mlsanity.com/paper/2506.03652/) (2025)
- [The Power of Many: Multi-Agent Multimodal Models for Cultural Image Captioning](https://mlsanity.com/paper/2411.11758/) (2024)
- [CompCap: Improving Multimodal Large Language Models with Composite Captions](https://mlsanity.com/paper/2412.05243/) (2024)
- [ArtSeek: Deep artwork understanding via multimodal in-context reasoning and late interaction retrieval](https://mlsanity.com/paper/2507.21917/) (2025)
- [KALE: An Artwork Image Captioning System Augmented with Heterogeneous Graph](https://mlsanity.com/paper/2409.10921/) (2024)
- [DenseAnnotate: Enabling Scalable Dense Caption Collection for Images and 3D Scenes via Spoken Descriptions](https://mlsanity.com/paper/2511.12452/) (2025)

## Abstract (arXiv)

Recent advances in vision-language models (VLMs) have improved image captioning for cultural heritage. However, inferring structured cultural metadata (e.g., creator, origin, period) from visual input remains underexplored. We introduce a multi-category, cross-cultural benchmark for this task and evaluate VLMs using an LLM-as-Judge framework that measures semantic alignment with reference annotations. To assess cultural reasoning, we report exact-match, partial-match, and attribute-level accuracy across cultural regions. Results show that models capture fragmented signals and exhibit substantial performance variation across cultures and metadata types, leading to inconsistent and weakly grounded predictions. These findings highlight the limitations of current VLMs in structured cultural metadata inference beyond visual perception.