mlsanityData to 14 Oct 2026

Pearl: A Multimodal Culturally-Aware Arabic Instruction Dataset

arXiv 2505.21979 ↗First posted 28 May 2025cs.CLcited by 6

On the map: Multimodal LLMs / vision-language models → MLLM training and visual instruction tuning

Abstract (arXiv)

Mainstream large vision-language models (LVLMs) inherently encode cultural biases, highlighting the need for diverse multimodal datasets. To address this gap, we introduce PEARL, a large-scale Arabic multimodal dataset and benchmark explicitly designed for cultural understanding. Constructed through advanced agentic workflows and extensive human-in-the-loop annotations by 37 annotators from across the Arab world, PEARL comprises over 309K multimodal examples spanning ten culturally significant domains covering all Arab countries. We further provide two robust evaluation benchmarks (PEARL and PEARL-LITE) along with a specialized subset (PEARL-X) explicitly developed to assess nuanced cultural variations. Comprehensive evaluations on state-of-the-art open and proprietary LVLMs demonstrate that reasoning-centric instruction alignment substantially improves models' cultural grounding compared to conventional scaling methods. PEARL establishes a foundational resource for advancing culturally-informed multimodal modeling research. All datasets and benchmarks are publicly available.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 25 of its 39 references that are in the map. First posted on arXiv 28 May 2025.

Builds on

25 of 39 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025Aya Vision: Advancing the Frontier of Multilingual Multimodality13 May
  2. 2024CAMEL-Bench: A Comprehensive Arabic LMM Benchmark24 Oct
  3. 2024Peacock: A Family of Arabic Multimodal Large Language Models and Benchmarks1 Mar
  4. 2024Exploring the Reasoning Abilities of Multimodal Large Language Models (MLLMs): A Comprehensive Survey on Emerging Trends in Multimodal Reasoning10 Jan
  5. 2023InfiMM-Eval: Complex Open-Ended Reasoning Evaluation For Multi-Modal Large Language Models20 Nov
  6. 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilities4 Aug

Other references · 19

  1. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  2. 2025Qwen2.5-VL Technical Reportcited by 5,587
  3. 2024SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilitiescited by 706
  4. 2024Visual CoT: Advancing Multi-Modal Language Models with a Comprehensive Dataset and Benchmark for Chain-of-Thought Reasoningcited by 280
  5. 2021Visually Grounded Reasoning across Languages and Culturescited by 189
  6. 2024CVQA: Culturally-diverse Multilingual Visual Question Answering Benchmarkcited by 120
  7. 2024Survey of Cultural Awareness in Language Models: Text and Beyondcited by 103
  8. 2021xGQA: Cross-Lingual Visual Question Answeringcited by 72
  9. 2025CultureVLM: Characterizing and Improving Cultural Understanding of Vision-Language Models for over 100 Countriescited by 42
  10. 2024M5 -- A Diverse Benchmark to Assess the Performance of Large Multimodal Models Across Multilingual and Multicultural Vision-Language Taskscited by 27
  11. 2024Evaluating Visual and Cultural Interpretation: The K-Viscuit Benchmark with Human-VLM Collaborationcited by 19
  12. 2024JMMMU: A Japanese Massive Multi-discipline Multimodal Understanding Benchmark for Culture-aware Evaluationcited by 19
7 more
  1. 2025GIMMICK -- Globally Inclusive Multimodal Multitask Cultural Knowledge Benchmarkingcited by 16
  2. 2024From Local Concepts to Universals: Evaluating the Multicultural Understanding of Vision-Language Modelscited by 14
  3. 2023HaVQA: A Dataset for Visual Question Answering and Multimodal Research in Hausa Languagecited by 12
  4. 2023Food-500 Cap: A Fine-Grained Food Caption Benchmark for Evaluating Vision-Language Modelscited by 10
  5. 2024ViTextVQA: A Large-Scale Visual Question Answering Dataset and a Novel Multimodal Feature Fusion Method for Vietnamese Text Comprehension in Imagescited by 7
  6. 2025JEEM: Vision-Language Understanding in Four Arabic Dialectscited by 7
  7. 2024How Culturally Aware are Vision-Language Models?cited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 4.

6 papers.

  1. 2025Grounding Multilingual Multimodal LLMs With Cultural Knowledgecited by 16
  2. 2025OASIS: A Multilingual and Multimodal Dataset for Culturally Grounded Spoken Visual QAcited by 5
  3. 2026Alexandria: A Multi-Domain Dialectal Arabic Machine Translation Dataset for Culturally Inclusive and Linguistically Diverse LLMscited by 1
  4. 2026Eye-Q: A Multilingual Benchmark for Visual Word Puzzle Solving and Image-to-Phrase Reasoningcited by 0
  5. 2026ImageEval 2026: Culturally Grounded Arabic Multimodal Evaluationcited by 0
  6. 2026CultureMINE: Datasets and Methods for Improving the Cultural Capabilities of NLP Systemscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025ARB: A Comprehensive Arabic Multimodal Reasoning Benchmark
  2. 2024All Languages Matter: Evaluating LMMs on Culturally Diverse 100 Languages
  3. 2025AIN: The Arabic INclusive Large Multimodal Model
  4. 2026CrossCult-KIBench: A Benchmark for Cross-Cultural Knowledge Insertion in MLLMs
  5. 2026Many Dialects, Many Languages, One Cultural Lens: Evaluating Multilingual VLMs for Bengali Culture Understanding Across Historically Linked Languages and Regional Dialects
  6. 2024Pangea: A Fully Open Multilingual Multimodal LLM for 39 Languages
  7. 2025EgMM-Corpus: A Multimodal Vision-Language Dataset for Egyptian Culture
  8. 2025Jawaher: A Multidialectal Dataset of Arabic Proverbs for LLM Benchmarking