mlsanityData to 7 Oct 2026

CultureVLM: Characterizing and Improving Cultural Understanding of Vision-Language Models for over 100 Countries

arXiv 2501.01282 ↗First posted 2 Jan 2025cs.AI cs.CL cs.CVcited by 40

On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning

Abstract (arXiv)

Vision-language models (VLMs) have advanced human-AI interaction but struggle with cultural understanding, often misinterpreting symbols, gestures, and artifacts due to biases in predominantly Western-centric training data. In this paper, we construct CultureVerse, a large-scale multimodal benchmark covering 19, 682 cultural concepts, 188 countries/regions, 15 cultural concepts, and 3 question types, with the aim of characterizing and improving VLMs' multicultural understanding capabilities. Then, we propose CultureVLM, a series of VLMs fine-tuned on our dataset to achieve significant performance improvement in cultural understanding. Our evaluation of 16 models reveals significant disparities, with a stronger performance in Western concepts and weaker results in African and Asian contexts. Fine-tuning on our CultureVerse enhances cultural perception, demonstrating cross-cultural, cross-continent, and cross-dataset generalization without sacrificing performance on models' general VLM benchmarks. We further present insights on cultural generalization and forgetting. We hope that this work could lay the foundation for more equitable and culturally aware multimodal AI systems.

Key facts. Cited by 40 papers in the map (data through 7 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 32 of its 52 references that are in the map. First posted on arXiv 2 Jan 2025.

Builds on

32 of 52 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024WorldCuisines: A Massive-Scale Benchmark for Multilingual and Multicultural Visual Question Answering on Global Cuisines16 Oct
  2. 2024Benchmarking Vision Language Models for Cultural Understanding15 Jul
  3. 2024CVQA: Culturally-diverse Multilingual Visual Question Answering Benchmark10 Jun

Other references · 29

  1. 2023Visual Instruction Tuningcited by 7,792
  2. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  3. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  4. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
  5. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
  6. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,883
  7. 2019Towards VQA Models That Can Readcited by 1,696
  8. 2024ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Toolscited by 1,096
  9. 2024MiniCPM-V: A GPT-4V Level MLLM on Your Phonecited by 1,061
  10. 2024A Comprehensive Survey of Hallucination Mitigation Techniques in Large Language Modelscited by 266
  11. 2023Assessing Cross-Cultural Alignment between ChatGPT and Human Societies: An Empirical Studycited by 203
  12. 2024BLEnD: A Benchmark for LLMs on Everyday Knowledge in Diverse Cultures and Languagescited by 171
17 more
  1. 2024Towards Measuring and Modeling "Culture" in LLMs: A Surveycited by 166
  2. 2023Having Beer after Prayer? Measuring Cultural Bias in Large Language Modelscited by 153
  3. 2023Cultural Alignment in Large Language Models: An Explanatory Analysis Based on Hofstede's Cultural Dimensionscited by 111
  4. 2022The Ghost in the Machine has an American accent: value conflict in GPT-3cited by 105
  5. 2023SeaLLMs -- Large Language Models for Southeast Asiacited by 104
  6. 2024CultureBank: An Online Community-Driven Knowledge Base Towards Culturally Aware Language Technologiescited by 98
  7. 2023Better to Ask in English: Cross-Lingual Evaluation of Large Language Models for Healthcare Queriescited by 82
  8. 2024WorldValuesBench: A Large-Scale Benchmark Dataset for Multi-Cultural Value Awareness of Language Modelscited by 59
  9. 2024CultureLLM: Incorporating Cultural Differences into Large Language Modelscited by 49
  10. 2022NormSAGE: Multi-Lingual Multi-Cultural Norm Discovery from Conversations On-the-Flycited by 48
  11. 2023Not All Countries Celebrate Thanksgiving: On the Cultural Dominance in Large Language Modelscited by 42
  12. 2023Are Multilingual LLMs Culturally-Diverse Reasoners? An Investigation into Multicultural Proverbs and Sayingscited by 32
  13. 2023Taiwan LLM: Bridging the Linguistic Divide with a Culturally Aligned Language Modelcited by 29
  14. 2023Typhoon: Thai Large Language Modelscited by 28
  15. 2024MM-Soc: Benchmarking Multimodal Large Language Models in Social Media Platformscited by 26
  16. 2024CulturePark: Boosting Cross-cultural Understanding in Large Language Modelscited by 24
  17. 2023ReadMe++: Benchmarking Multilingual Language Models for Multi-Domain Readability Assessmentcited by 19

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 27; 2026: 13.

40 papers.

Sort:

By area:

  1. 2025Large Language Model Agent: A Survey on Methodology, Applications and Challengescited by 194
  2. 2025Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systemscited by 125
  3. 2025Grounding Multilingual Multimodal LLMs With Cultural Knowledgecited by 16
  4. 2025Diffusion Models Through a Global Lens: Are They Culturally Inclusive?cited by 14
  5. 2025Time Travel: A Comprehensive Benchmark to Evaluate LMMs on Historical and Cultural Artifactscited by 10
  6. 2025SARA: Selective and Adaptive Retrieval-augmented Generation with Context Compressioncited by 9
  7. 2025Culture in Action: Evaluating Text-to-Image Models through Social Activitiescited by 9
  8. 2025RAVENEA: A Benchmark for Multimodal Retrieval-Augmented Visual Culture Understandingcited by 8
  9. 2025Multimodal Cultural Safety: Evaluation Framework and Alignment Strategiescited by 8
  10. 2025Seeing Culture: A Benchmark for Visual Reasoning and Groundingcited by 8
  11. 2025Reasoning Is Not All You Need: Examining LLMs for Multi-Turn Mental Health Conversationscited by 6
  12. 2025Pearl: A Multimodal Culturally-Aware Arabic Instruction Datasetcited by 6
28 more
  1. 2025Topological Structure Learning Should Be A Research Priority for LLM-Based Multi-Agent Systemscited by 5
  2. 2025Hanfu-Bench: A Multimodal Benchmark on Cross-Temporal Cultural Understanding and Transcreationcited by 5
  3. 2025VIBE: Can a VLM Read the Room?cited by 4
  4. 2025Evaluation of Cultural Competence of Vision-Language Modelscited by 3
  5. 2025CultureCLIP: Empowering CLIP with Cultural Awareness through Synthetic Images and Contextualized Captionscited by 3
  6. 2025Seeing Symbols, Missing Cultures: Probing Vision-Language Models' Reasoning on Fire Imagery and Cultural Meaningcited by 3
  7. 2025World in a Frame: Understanding Culture Mixing as a New Challenge for Vision-Language Modelscited by 3
  8. 2026Many Dialects, Many Languages, One Cultural Lens: Evaluating Multilingual VLMs for Bengali Culture Understanding Across Historically Linked Languages and Regional Dialectscited by 3
  9. 2025Domain Specific Benchmarks for Evaluating Multimodal Large Language Modelscited by 2
  10. 2025Bias Beyond Demographics: Probing Decision Boundaries in Black-Box LVLMs via Counterfactual VQAcited by 2
  11. 2025Vision Language Models are Confused Touristscited by 2
  12. 2026VULCA-Bench: A Multicultural Vision-Language Benchmark for Evaluating Cultural Understandingcited by 2
  13. 2026From Native Memes to Global Moderation: Cross-Cultural Evaluation of Vision-Language Models for Hateful Meme Detectioncited by 2
  14. 2025ScreenLLM: Stateful Screen Schema for Efficient Action Understanding and Predictioncited by 1
  15. 2025Toward Socially Aware Vision-Language Models: Evaluating Cultural Competence Through Multimodal Story Generationcited by 1
  16. 2025DenseAnnotate: Enabling Scalable Dense Caption Collection for Images and 3D Scenes via Spoken Descriptionscited by 1
  17. 2025Are Vision Language Models Cross-Cultural Theory of Mind Reasoners?cited by 1
  18. 2026Unlocking Cognitive Capabilities and Analyzing the Perception-Logic Trade-offcited by 1
  19. 2026HanMoVLM: Large Vision-Language Models for Professional Artistic Painting Evaluationcited by 1
  20. 2026When Cultures Move: Measuring and Improving Multicultural Text-to-Video Generationcited by 1
  21. 2026Jury Duty: Calibration and Orientation Failures in MLLM-as-a-Judge Under Cultural Ambiguitycited by 1
  22. 2026Failing to See or Failing to Know? Attributing Errors in Vision-Language Modelscited by 1
  23. 2025Labels or Input? Rethinking Augmentation in Multimodal Hate Detectioncited by 0
  24. 2026Appear2Meaning: A Cross-Cultural Benchmark for Structured Cultural Metadata Inference from Imagescited by 0
  25. 2026On the Cultural Anachronism and Temporal Reasoning in Vision Language Modelscited by 0
  26. 2026KG-FairDiff: Knowledge Graph-Guided Prompt Refinement for Demographically Fair Text-to-Image Generationcited by 0
  27. 2026Cultural Moment Benchmark: Evaluating Video Cultural Reasoning and Grounding in Southeast Asiacited by 0
  28. 2026NormViz: A Benchmark and Framework for Grounding Multimodal Reasoning in Global Culturescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025BLEnD-Vis: Benchmarking Multimodal Cultural Understanding in Vision Language Models
  2. 2024Evaluating Visual and Cultural Interpretation: The K-Viscuit Benchmark with Human-VLM Collaboration
  3. 2025IndicVisionBench: Benchmarking Cultural and Multilingual Understanding in VLMs
  4. 2025Rice-VL: Evaluating Vision-Language Models for Cultural Understanding Across ASEAN Countries
  5. 2024CROPE: Evaluating In-Context Adaptation of Vision and Language Models to Culture-Specific Concepts
  6. 2024From Local Concepts to Universals: Evaluating the Multicultural Understanding of Vision-Language Models
  7. 2025Uncovering Cultural Representation Disparities in Vision-Language Models
  8. 2025Culture In a Frame: C3B as a Comic-Based Benchmark for Multimodal Culturally Awareness