mlsanityData to 14 Oct 2026

Grounding Multilingual Multimodal LLMs With Cultural Knowledge

arXiv 2508.07414 ↗First posted 10 Aug 2025cs.CL cs.LGcited by 16

On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning

Abstract (arXiv)

Multimodal Large Language Models excel in high-resource settings, but often misinterpret long-tail cultural entities and underperform in low-resource languages. To address this gap, we propose a data-centric approach that directly grounds MLLMs in cultural knowledge. Leveraging a large scale knowledge graph from Wikidata, we collect images that represent culturally significant entities, and generate synthetic multilingual visual question answering data. The resulting dataset, CulturalGround, comprises 22 million high-quality, culturally-rich VQA pairs spanning 42 countries and 39 languages. We train an open-source MLLM CulturalPangea on CulturalGround, interleaving standard multilingual instruction-tuning data to preserve general abilities. CulturalPangea achieves state-of-the-art performance among open models on various culture-focused multilingual multimodal benchmarks, outperforming prior models by an average of 5.0 without degrading results on mainstream vision-language tasks. Our findings show that our targeted, culturally grounded approach could substantially narrow the cultural gap in MLLMs and offer a practical path towards globally inclusive multimodal systems.

Key facts. Cited by 16 papers in the map (data through 14 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 29 of its 46 references that are in the map. First posted on arXiv 10 Aug 2025.

Builds on

29 of 46 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025CultureVLM: Characterizing and Improving Cultural Understanding of Vision-Language Models for over 100 Countries2 Jan
  2. 2024All Languages Matter: Evaluating LMMs on Culturally Diverse 100 Languages25 Nov
  3. 2023mBLIP: Efficient Bootstrapping of Multilingual Vision-LLMs13 Jul

Other references · 26

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  3. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  4. 2025Gemma 3 Technical Reportcited by 1,824
  5. 2018Experience Replay for Continual Learningcited by 1,095
  6. 2023Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunchcited by 694
  7. 2019KEPLER: A Unified Model for Knowledge Embedding and Pre-trained Language Representationcited by 356
  8. 2019K-BERT: Enabling Language Representation with Knowledge Graphcited by 304
  9. 2021WIT: Wikipedia-based Image Text Dataset for Multimodal Multilingual Machine Learningcited by 287
  10. 2019Massively Multilingual Neural Machine Translation in the Wild: Findings and Challengescited by 257
  11. 2021New Insights on Reducing Abrupt Representation Change in Online Continual Learningcited by 219
  12. 2022Challenges and Strategies in Cross-Cultural NLPcited by 218
14 more
  1. 2023Do All Languages Cost the Same? Tokenization in the Era of Commercial Language Modelscited by 192
  2. 2021Visually Grounded Reasoning across Languages and Culturescited by 189
  3. 2022SODA: Million-scale Dialogue Distillation with Social Commonsense Contextualizationcited by 168
  4. 2024Investigating Cultural Alignment of Large Language Modelscited by 143
  5. 2021Systematic Inequalities in Language Technology Performance across the World's Languagescited by 120
  6. 2020X-FACTR: Multilingual Factual Knowledge Retrieval from Pretrained Language Modelscited by 117
  7. 2022Crossmodal-3600: A Massively Multilingual Multimodal Evaluation Datasetcited by 100
  8. 2024Pangea: A Fully Open Multilingual Multimodal LLM for 39 Languagescited by 64
  9. 2022Continual evaluation for lifelong learning: Identifying the stability gapcited by 56
  10. 2023GeoDE: a Geographically Diverse Evaluation Dataset for Object Recognitioncited by 50
  11. 2022Beyond Counting Datasets: A Survey of Multilingual Dataset Construction and Necessary Resourcescited by 25
  12. 2023Large Scale Multi-Lingual Multi-Modal Summarization Datasetcited by 12
  13. 2025Crowdsource, Crawl, or Generate? Creating SEA-VL, a Multicultural Vision-Language Dataset for Southeast Asiacited by 9
  14. 2025Pearl: A Multimodal Culturally-Aware Arabic Instruction Datasetcited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 13.

16 papers.

Sort:

By area:

  1. 2025Finding Culture-Sensitive Neurons in Vision-Language Modelscited by 3
  2. 2026SEA-Guard: Culturally Grounded Multilingual Safeguard for Southeast Asiacited by 2
  3. 2026Anthropogenic Regional Adaptation in Multimodal Vision-Language Modelcited by 2
  4. 2025TowerVision: Understanding and Improving Multilinguality in Vision-Language Modelscited by 1
  5. 2026CultureVidBench: Benchmarking Cultural Understanding in Text-to-Video Generationcited by 1
  6. 2026Multi-turn Conversational AI from Text to Multimodal Interaction: Data, Models, Evaluation, and Open Challengescited by 1
  7. 2025The Curious Case of Curiosity across Human Cultures and LLMscited by 0
  8. 2026PinCLIP: Large-scale Foundational Multimodal Representation at Pinterestcited by 0
  9. 2026Toward Culturally Grounded Natural Language Processingcited by 0
  10. 2026Mimir: Large-scale Multilingual Concept Modelingcited by 0
  11. 2026Sci-Rho: A Multilingual Visually-Grounded Symbolic Benchmark for STEM Problemscited by 0
  12. 2026ArtECulture: Benchmarking Culture-Conditioned Visual Emotion Understanding in Multimodal Large Language Modelscited by 0
4 more
  1. 2026VFA: Empowering Multilingual MLLMs via Vision-Free Adaptationcited by 0
  2. 2026Think Before You Link: Rarity, Reasoning, and Retrieval in Multilingual Entity Linkingcited by 0
  3. 2026CultureMINE: Datasets and Methods for Improving the Cultural Capabilities of NLP Systemscited by 0
  4. 2026SEA-CLIP-Tiny: Efficient Multilingual Text-Vision Embedding for Southeast Asian Languagescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025MMA-ASIA: A Multilingual and Multimodal Alignment Framework for Culturally-Grounded Evaluation
  2. 2025Traveling Across Languages: Benchmarking Cross-Lingual Consistency in Multimodal LLMs
  3. 2025BLEnD-Vis: Benchmarking Multimodal Cultural Understanding in Vision Language Models
  4. 2025MELLA: Bridging Linguistic Capability and Cultural Groundedness for Low-Resource Language MLLMs
  5. 2025M4-RAG: A Massive-Scale Multilingual Multi-Cultural Multimodal RAG
  6. 2025Rice-VL: Evaluating Vision-Language Models for Cultural Understanding Across ASEAN Countries
  7. 2025NativQA Framework: Enabling LLMs and VLMs with Native, Local, and Everyday Knowledge
  8. 2025IndicVisionBench: Benchmarking Cultural and Multilingual Understanding in VLMs