mlsanityData to 7 Oct 2026

Peacock: A Family of Arabic Multimodal Large Language Models and Benchmarks

arXiv 2403.01031 ↗First posted 1 Mar 2024cs.CL cs.AIcited by 32

On the map: Multimodal LLMs / vision-language models → MLLM training and visual instruction tuning

Abstract (arXiv)

Multimodal large language models (MLLMs) have proven effective in a wide range of tasks requiring complex reasoning and linguistic comprehension. However, due to a lack of high-quality multimodal resources in languages other than English, success of MLLMs remains relatively limited to English-based settings. This poses significant challenges in developing comparable models for other languages, including even those with large speaker populations such as Arabic. To alleviate this challenge, we introduce a comprehensive family of Arabic MLLMs, dubbed textitPeacock, with strong vision and language capabilities. Through comprehensive qualitative and quantitative analysis, we demonstrate the solid performance of our models on various visual reasoning tasks and further show their emerging dialectal potential. Additionally, we introduce ~textitHenna, a new benchmark specifically designed for assessing MLLMs on aspects related to Arabic culture, setting the first stone for culturally-aware Arabic MLLMs.The GitHub repository for the textitPeacock project is available at urlhttps://github.com/UBC-NLP/peacock.

Key facts. Cited by 32 papers in the map (data through 7 Oct 2026). Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 32 of its 56 references that are in the map. First posted on arXiv 1 Mar 2024.

Builds on

32 of 56 references.

Closest ancestors · 4

Same direction, newest first

  1. 2023Improved Baselines with Visual Instruction Tuning5 Oct
  2. 2023The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)29 Sep
  3. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning11 May
  4. 2023Otter: A Multi-Modal Model with In-Context Instruction Tuning5 May

Other references · 28

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  6. 2023GPT-4 Technical Reportcited by 18,132
  7. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  8. 2022Training language models to follow instructions with human feedbackcited by 15,712
  9. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  10. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  11. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  12. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
16 more
  1. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,498
  2. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,544
  3. 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,290
  4. 2021Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Conceptscited by 1,209
  5. 2019OK-VQA: A Visual Question Answering Benchmark Requiring External Knowledgecited by 1,114
  6. 2020Language-agnostic BERT Sentence Embeddingcited by 621
  7. 2023MIMIC-IT: Multi-Modal In-Context Instruction Tuningcited by 264
  8. 2023Platypus: Quick, Cheap, and Powerful Refinement of LLMscited by 150
  9. 2022MultiInstruct: Improving Multi-Modal Zero-Shot Learning via Instruction Tuningcited by 109
  10. 2021Colossal-AI: A Unified Deep Learning System For Large-Scale Parallel Trainingcited by 96
  11. 2023mBLIP: Efficient Bootstrapping of Multilingual Vision-LLMscited by 41
  12. 20161.5 billion words Arabic Corpuscited by 35
  13. 2020Machine Generation and Detection of Arabic Manipulated and Fake Newscited by 16
  14. 2023TARJAMAT: Evaluation of Bard and ChatGPT on Machine Translation of Ten Arabic Varietiescited by 16
  15. 2023Violet: A Vision-Language Model for Arabic Image Captioning with Gemini Decodercited by 7
  16. 2022Bench-Marking And Improving Arabic Automatic Image Captioning Through The Use Of Multi-Task Learning Paradigmcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 10; 2025: 19; 2026: 3.

32 papers.

Sort:

By area:

  1. 2024All Languages Matter: Evaluating LMMs on Culturally Diverse 100 Languagescited by 55
  2. 2024A Survey on Evaluation of Multimodal Large Language Modelscited by 46
  3. 2024CaLMQA: Exploring culturally specific long-form question answering across 23 languagescited by 31
  4. 2024A Survey on Multimodal Benchmarks: In the Era of Large AI Modelscited by 27
  5. 2025PalmX 2025: The First Shared Task on Benchmarking LLMs on Arabic and Islamic Culturecited by 20
  6. 2024CAMEL-Bench: A Comprehensive Arabic LMM Benchmarkcited by 17
  7. 2024Survey of Large Multimodal Model Datasets, Application Categories and Taxonomycited by 17
  8. 2024Dallah: A Dialect-Aware Multimodal Large Language Model for Arabiccited by 16
  9. 2025Palm: A Culturally Inclusive and Linguistically Diverse Dataset for Arabic LLMscited by 15
  10. 2025KITAB-Bench: A Comprehensive Multi-Domain Benchmark for Arabic OCR and Document Understandingcited by 14
  11. 2025The Landscape of Arabic Large Language Models (ALLMs): A New Era for Arabic Language Technologycited by 12
  12. 2025Time Travel: A Comprehensive Benchmark to Evaluate LMMs on Historical and Cultural Artifactscited by 10
20 more
  1. 2025Evaluating Arabic Large Language Models: A Survey of Benchmarks, Methods, and Gapscited by 9
  2. 2024Qalam : A Multimodal LLM for Arabic Optical Character and Handwriting Recognitioncited by 7
  3. 2025JEEM: Vision-Language Understanding in Four Arabic Dialectscited by 7
  4. 2025IndicVisionBench: Benchmarking Cultural and Multilingual Understanding in VLMscited by 7
  5. 2025LLM Alignment for the Arabs: A Homogenous Culture or Diverse Ones?cited by 6
  6. 2025Pearl: A Multimodal Culturally-Aware Arabic Instruction Datasetcited by 6
  7. 2024LaVy: Vietnamese Multimodal Large Language Modelcited by 5
  8. 2025QARI-OCR: High-Fidelity Arabic Text Recognition through Multimodal Large Language Model Adaptationcited by 5
  9. 2025Hala Technical Report: Building Arabic-Centric Instruction & Translation Models at Scalecited by 5
  10. 2025Large Multimodal Models for Low-Resource Languages: A Surveycited by 4
  11. 2025ARB: A Comprehensive Arabic Multimodal Reasoning Benchmarkcited by 4
  12. 2025Isolating Culture Neurons in Multilingual Large Language Modelscited by 4
  13. 2025From Words to Proverbs: Evaluating LLMs Linguistic and Cultural Competence in Saudi Dialects with Abshercited by 2
  14. 2024Arabic Automatic Story Generation with Large Language Modelscited by 1
  15. 2025Arabic Multimodal Machine Learning: Datasets, Applications, Approaches, and Challengescited by 1
  16. 2026ALPS: A Diagnostic Challenge Set for Arabic Linguistic & Pragmatic Reasoningcited by 1
  17. 2025VQA support to Arabic Language Learning Educational Toolcited by 0
  18. 2025Wasm: A Pipeline for Constructing Structured Arabic Interleaved Multimodal Corporacited by 0
  19. 2026Cultural Benchmarking of LLMs in Standard and Dialectal Arabic Dialoguescited by 0
  20. 2026CultureMINE: Datasets and Methods for Improving the Cultural Capabilities of NLP Systemscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025AIN: The Arabic INclusive Large Multimodal Model
  2. 2024Efficient Multimodal Large Language Models: A Survey
  3. 2024A Survey on Benchmarks of Multimodal Large Language Models
  4. 2024The Revolution of Multimodal Large Language Models: A Survey
  5. 2023SEED-Bench-2: Benchmarking Multimodal Large Language Models
  6. 2024A Survey of Multimodal Large Language Model from A Data-centric Perspective
  7. 2024A Comprehensive Review of Multimodal Large Language Models: Performance and Challenges Across Different Tasks
  8. 2024A Comprehensive Survey and Guide to Multimodal Large Language Models in Vision-Language Tasks