mlsanityData to 7 Oct 2026

Dallah: A Dialect-Aware Multimodal Large Language Model for Arabic

arXiv 2407.18129 ↗First posted 25 Jul 2024cs.CL cs.AIcited by 16

On the map: Multimodal LLMs / vision-language models → MLLM training and visual instruction tuning

Abstract (arXiv)

Recent advancements have significantly enhanced the capabilities of Multimodal Large Language Models (MLLMs) in generating and understanding image-to-text content. Despite these successes, progress is predominantly limited to English due to the scarcity of high quality multimodal resources in other languages. This limitation impedes the development of competitive models in languages such as Arabic. To alleviate this situation, we introduce an efficient Arabic multimodal assistant, dubbed Dallah, that utilizes an advanced language model based on LLaMA-2 to facilitate multimodal interactions. Dallah demonstrates state-of-the-art performance in Arabic MLLMs. Through fine-tuning six Arabic dialects, Dallah showcases its capability to handle complex dialectal interactions incorporating both textual and visual elements. The model excels in two benchmark tests: one evaluating its performance on Modern Standard Arabic (MSA) and another specifically designed to assess dialectal responses. Beyond its robust performance in multimodal interaction tasks, Dallah has the potential to pave the way for further development of dialect-aware Arabic MLLMs.

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 21 of its 38 references that are in the map. First posted on arXiv 25 Jul 2024.

Builds on

21 of 38 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024Peacock: A Family of Arabic Multimodal Large Language Models and Benchmarks1 Mar
  2. 2024PALO: A Polyglot Large Multimodal Model for 5B People22 Feb
  3. 2023ShareGPT4V: Improving Large Multi-Modal Models with Better Captions21 Nov
  4. 2023Improved Baselines with Visual Instruction Tuning5 Oct

Other references · 17

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2023GPT-4 Technical Reportcited by 18,132
  6. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  7. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  8. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  9. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  10. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  11. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  12. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,883
5 more
  1. 2024Multilingual E5 Text Embeddings: A Technical Reportcited by 357
  2. 2024ALLaVA: Harnessing GPT4V-Synthesized Data for Lite Vision-Language Modelscited by 204
  3. 2023To See is to Believe: Prompting GPT-4V for Better Visual Instruction Tuningcited by 143
  4. 2023AceGPT, Localizing Large Language Models in Arabiccited by 75
  5. 2023Violet: A Vision-Language Model for Arabic Image Captioning with Gemini Decodercited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 8; 2026: 6.

16 papers.

Sort:

By area:

  1. 2024AraDiCE: Benchmarks for Dialectal and Cultural Capabilities in LLMscited by 46
  2. 2025MultiLoKo: a multilingual local knowledge benchmark for LLMs spanning 31 languagescited by 22
  3. 2025The Landscape of Arabic Large Language Models (ALLMs): A New Era for Arabic Language Technologycited by 12
  4. 2024Swan and ArabicMTEB: Dialect-Aware, Arabic-Centric, Cross-Lingual, and Cross-Cultural Embedding Models and Benchmarkscited by 10
  5. 2025Large Multimodal Models for Low-Resource Languages: A Surveycited by 4
  6. 2025Fine-Tuning LLMs for Low-Resource Dialect Translation: The Case of Lebanesecited by 2
  7. 2025From Words to Proverbs: Evaluating LLMs Linguistic and Cultural Competence in Saudi Dialects with Abshercited by 2
  8. 2026Once Correct, Still Wrong: Counterfactual Hallucination in Multilingual Vision-Language Modelscited by 2
  9. 2026Said Aloud, Read Different: Cross-Modal Instability in Multimodal Modelscited by 2
  10. 2025Arabic Multimodal Machine Learning: Datasets, Applications, Approaches, and Challengescited by 1
  11. 2026dziribot: rag based intelligent conversational agent for algerian arabic dialectcited by 1
  12. 2026Multi-turn Conversational AI from Text to Multimodal Interaction: Data, Models, Evaluation, and Open Challengescited by 1
4 more
  1. 2025VQA support to Arabic Language Learning Educational Toolcited by 0
  2. 2025The Rise of AfricaNLP: A Survey of Contributions, Contributors, Community Impact, and Bibliometric Analysiscited by 0
  3. 2026DuwatBench: Bridging Language and Visual Heritage through an Arabic Calligraphy Benchmark for Multimodal Understandingcited by 0
  4. 2026ImageEval 2026: Culturally Grounded Arabic Multimodal Evaluationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025AIN: The Arabic INclusive Large Multimodal Model
  2. 2025DialectalArabicMMLU: Benchmarking Dialectal Capabilities in Arabic and Multilingual Language Models
  3. 2024The Revolution of Multimodal Large Language Models: A Survey
  4. 2024A Comprehensive Survey and Guide to Multimodal Large Language Models in Vision-Language Tasks
  5. 2024A Comprehensive Review of Multimodal Large Language Models: Performance and Challenges Across Different Tasks
  6. 2024A Survey of Multimodal Large Language Model from A Data-centric Perspective
  7. 2024Efficient Multimodal Large Language Models: A Survey
  8. 2025ARB: A Comprehensive Arabic Multimodal Reasoning Benchmark