mlsanityData to 14 Oct 2026

A Survey of Large Language Models for Arabic Language and its Dialects

arXiv 2410.20238 ↗First posted 26 Oct 2024cs.CL cs.AIcited by 13

Not on the map. Why?

Abstract (arXiv)

This survey offers a comprehensive overview of Large Language Models (LLMs) designed for Arabic language and its dialects. It covers key architectures, including encoder-only, decoder-only, and encoder-decoder models, along with the datasets used for pre-training, spanning Classical Arabic, Modern Standard Arabic, and Dialectal Arabic. The study also explores monolingual, bilingual, and multilingual LLMs, analyzing their architectures and performance across downstream tasks, such as sentiment analysis, named entity recognition, and question answering. Furthermore, it assesses the openness of Arabic LLMs based on factors, such as source code availability, training data, model weights, and documentation. The survey highlights the need for more diverse dialectal datasets and attributes the importance of openness for research reproducibility and transparency. It concludes by identifying key challenges and opportunities for future research and stressing the need for more inclusive and representative models.

Key facts. Cited by 13 papers in the map (data through 14 Oct 2026). Builds on 46 of its 130 references that are in the map. First posted on arXiv 26 Oct 2024.

Builds on

46 of 130 references.

References · 46

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  5. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  6. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  7. 2024GPT-4o System Cardcited by 4,588
  8. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
  9. 2023A Survey of Large Language Modelscited by 2,576
  10. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,998
  11. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,989
  12. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,818
34 more
  1. 2020Multilingual Denoising Pre-training for Neural Machine Translationcited by 1,035
  2. 2020Language-agnostic BERT Sentence Embeddingcited by 627
  3. 2024OLMo: Accelerating the Science of Language Modelscited by 613
  4. 2019Defending Against Neural Fake Newscited by 559
  5. 2020AraBERT: Transformer-based Model for Arabic Language Understandingcited by 362
  6. 2024Aya Model: An Instruction Finetuned Open-Access Multilingual Language Modelcited by 295
  7. 2019Leveraging Pre-trained Checkpoints for Sequence Generation Taskscited by 182
  8. 2020ARBERT & MARBERT: Deep Bidirectional Transformers for Arabiccited by 166
  9. 2022mGPT: Few-Shot Learners Go Multilingualcited by 142
  10. 2021The Interplay of Variant, Size, and Task Type in Arabic Pre-trained Language Modelscited by 119
  11. 2024Stable LM 2 1.6B Technical Reportcited by 90
  12. 2020KUISAIL at SemEval-2020 Task 12: BERT-CNN for Offensive Speech Identification in Social Mediacited by 84
  13. 2023AceGPT, Localizing Large Language Models in Arabiccited by 75
  14. 2023Jais and Jais-chat: Arabic-Centric Foundation and Instruction-Tuned Open Generative Large Language Modelscited by 69
  15. 2021AraT5: Text-to-Text Transformers for Arabic Language Generationcited by 63
  16. 2020AraGPT2: Pre-Trained Transformer for Arabic Language Generationcited by 55
  17. 2024ALLaM: Large Language Models for Arabic and Englishcited by 55
  18. 2020AraELECTRA: Pre-Training Text Discriminators for Arabic Language Understandingcited by 43
  19. 2022AraBART: a Pretrained Arabic Sequence-to-Sequence Model for Abstractive Summarizationcited by 24
  20. 2018Arap-Tweet: A Large Multi-Dialect Twitter Corpus for Gender, Age and Language Variety Identificationcited by 23
  21. 2024The Model Openness Framework: Promoting Completeness and Openness for Reproducibility, Transparency, and Usability in Artificial Intelligencecited by 23
  22. 2021DziriBERT: a Pre-trained Language Model for the Algerian Dialectcited by 18
  23. 2023TARJAMAT: Evaluation of Bard and ChatGPT on Machine Translation of Ten Arabic Varietiescited by 16
  24. 2020Multi-Dialect Arabic BERT for Country-Level Dialect Identificationcited by 15
  25. 2024Atlas-Chat: Adapting Large Language Models for Low-Resource Moroccan Arabic Dialectcited by 13
  26. 2022JASMINE: Arabic GPT Models for Few-Shot Learningcited by 11
  27. 2024CamelEval: Advancing Culturally Aligned Arabic Language Models and Benchmarkscited by 10
  28. 2024AraPoemBERT: A Pretrained Language Model for Arabic Poetry Analysiscited by 8
  29. 2022AraLegal-BERT: A pretrained language model for Arabic Legal textcited by 7
  30. 2023AraMUS: Pushing the Limits of Data and Model Scale for Arabic Natural Language Processingcited by 7
  31. 2024ArabianGPT: Native Arabic GPT-based Large Language Modelcited by 7
  32. 2024SaudiBERT: A Large Language Model Pretrained on Saudi Dialect Corporacited by 7
  33. 2024EgyBERT: A Large Language Model Pretrained on Egyptian Dialect Corporacited by 5
  34. 2024Arabic Automatic Story Generation with Large Language Modelscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 4; 2026: 9.

13 papers.

Sort:

By area:

  1. 2025The Landscape of Arabic Large Language Models (ALLMs): A New Era for Arabic Language Technologycited by 12
  2. 2025Evaluating Arabic Large Language Models: A Survey of Benchmarks, Methods, and Gapscited by 9
  3. 2026HalluScore: Large Language Model Hallucination Question Answering Benchmarkcited by 2
  4. 2025chDzDT: Word-level morphology-aware language model for Algerian social media textcited by 1
  5. 2026Grounding Arabic LLMs in the Doha Historical Dictionary: Retrieval-Augmented Understanding of Quran and Hadithcited by 1
  6. 2025MAPROC at AHaSIS Shared Task: Few-Shot and Sentence Transformer for Sentiment Analysis of Arabic Hotel Reviewscited by 0
  7. 2026Arabic Prompts with English Tools: A Benchmarkcited by 0
  8. 2026Are Arabic Benchmarks Reliable? QIMMA's Quality-First Approach to LLM Evaluationcited by 0
  9. 2026State-of-the-Art Arabic Language Modeling with Sparse MoE Fine-Tuning and Chain-of-Thought Distillationcited by 0
  10. 2026Evaluation of Small Language Models for Arabic Language Processingcited by 0
  11. 2026ArabicDialectSafety: A Dialect-Aware Benchmark for Arabic Content Safety Classificationcited by 0
  12. 2026A Comprehensive Analysis of Arabic Natural Language Processing Research: Trends, Topic Evolution, and Research Gaps -- A Bibliometric and Topic-Based Studycited by 0
1 more
  1. 2026Halluscoring 2026: The first shared task on llms hallucination detection and answer verificationcited by 0