mlsanityData to 7 Oct 2026

Culture In a Frame: C3B as a Comic-Based Benchmark for Multimodal Culturally Awareness

arXiv 2510.00041 ↗First posted 27 Sep 2025cs.CV cs.AIcited by 2

On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning

Abstract (arXiv)

Cultural awareness capabilities have emerged as a critical capability for Multimodal Large Language Models (MLLMs). However, current benchmarks lack progressed difficulty in their task design and are deficient in cross-lingual tasks. Moreover, current benchmarks often use real-world images. Each real-world image typically contains one culture, making these benchmarks relatively easy for MLLMs. Based on this, we propose C3B (Comics Cross-Cultural Benchmark), a novel multicultural, multitask and multilingual cultural awareness capabilities benchmark. C3B comprises over 2000 images and over 18000 QA pairs, constructed on three tasks with progressed difficulties, from basic visual recognition to higher-level cultural conflict understanding, and finally to cultural content generation. We conducted evaluations on 11 open-source MLLMs, revealing a significant performance gap between MLLMs and human performance. The gap demonstrates that C3B poses substantial challenges for current MLLMs, encouraging future research to advance the cultural awareness capabilities of MLLMs.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 32 of its 42 references that are in the map. First posted on arXiv 27 Sep 2025.

Builds on

32 of 42 references.

Closest ancestors · 8

Same direction, newest first

  1. 2025TCC-Bench: Benchmarking the Traditional Chinese Culture Understanding Capabilities of MLLMs16 May
  2. 2025GIMMICK -- Globally Inclusive Multimodal Multitask Cultural Knowledge Benchmarking19 Feb
  3. 2024All Languages Matter: Evaluating LMMs on Culturally Diverse 100 Languages25 Nov
  4. 2024Benchmarking Vision Language Models for Cultural Understanding15 Jul
  5. 2024Vision-Language Models under Cultural and Inclusive Considerations8 Jul
  6. 2024From Local Concepts to Universals: Evaluating the Multicultural Understanding of Vision-Language Models28 Jun
  7. 2024How Culturally Aware are Vision-Language Models?24 May
  8. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond24 Aug

Other references · 24

  1. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  2. 2025Qwen2.5-VL Technical Reportcited by 5,418
  3. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  4. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
  5. 2020BLEURT: Learning Robust Metrics for Text Generationcited by 1,196
  6. 2023MiniGPT-v2: large language model as a unified interface for vision-language multi-task learningcited by 617
  7. 2023Monkey: Image Resolution and Text Label Are Important Things for Large Multi-modal Modelscited by 356
  8. 2015Sketch-based Manga Retrieval using Manga109 Datasetcited by 326
  9. 2023DreamLLM: Synergistic Multimodal Comprehension and Creationcited by 296
  10. 2024mPLUG-Owl3: Towards Long Image-Sequence Understanding in Multi-Modal Large Language Modelscited by 285
  11. 2023SPHINX: The Joint Mixing of Weights, Tasks, and Visual Embeddings for Multi-modal Large Language Modelscited by 283
  12. 2024InternLM-XComposer-2.5: A Versatile Large Vision Language Model Supporting Long-Contextual Input and Outputcited by 195
12 more
  1. 2024Global MMLU: Understanding and Addressing Cultural and Linguistic Biases in Multilingual Evaluationcited by 181
  2. 2023Having Beer after Prayer? Measuring Cultural Bias in Large Language Modelscited by 153
  3. 2024Investigating Cultural Alignment of Large Language Modelscited by 140
  4. 2024The Revolution of Multimodal Large Language Models: A Surveycited by 134
  5. 2024BetterBench: Assessing AI Benchmarks, Uncovering Issues, and Establishing Best Practicescited by 112
  6. 2016The Amazing Mysteries of the Gutter: Drawing Inferences Between Panels in Comic Book Narrativescited by 41
  7. 2024CaLMQA: Exploring culturally specific long-form question answering across 23 languagescited by 31
  8. 2025Visual question answering: from early developments to recent advances -- a surveycited by 19
  9. 2024CoMix: A Comprehensive Benchmark for Multi-Task Comic Understandingcited by 15
  10. 2025Cultural Bias Matters: A Cross-Cultural Benchmark Dataset and Sentiment-Enriched Model for Understanding Multimodal Metaphorscited by 10
  11. 2024Make Imagination Clearer! Stable Diffusion-based Visual Imagination for Multimodal Machine Translationcited by 5
  12. 2025RONA: Pragmatically Diverse Image Captioning with Coherence Relationscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026XCR-Bench: Benchmarking Cross-Cultural Reasoning in LLMs via Culture-Specific Items and Hall's Triadcited by 2
  2. 2026VFA: Empowering Multilingual MLLMs via Vision-Free Adaptationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025CultureVLM: Characterizing and Improving Cultural Understanding of Vision-Language Models for over 100 Countries
  2. 2025MMA-ASIA: A Multilingual and Multimodal Alignment Framework for Culturally-Grounded Evaluation
  3. 2025Toward Socially Aware Vision-Language Models: Evaluating Cultural Competence Through Multimodal Story Generation
  4. 2026CrossCult-KIBench: A Benchmark for Cross-Cultural Knowledge Insertion in MLLMs
  5. 2025Grounding Multilingual Multimodal LLMs With Cultural Knowledge
  6. 2025Seeing Culture: A Benchmark for Visual Reasoning and Grounding
  7. 2024Exploring Visual Culture Awareness in GPT-4V: A Comprehensive Probing
  8. 2026VULCA-Bench: A Multicultural Vision-Language Benchmark for Evaluating Cultural Understanding