mlsanityData to 7 Oct 2026

Jais and Jais-chat: Arabic-Centric Foundation and Instruction-Tuned Open Generative Large Language Models

arXiv 2308.16149 ↗First posted 30 Aug 2023cs.CL cs.AI cs.LGcited by 68

Not on the map. Why?

Abstract (arXiv)

We introduce Jais and Jais-chat, new state-of-the-art Arabic-centric foundation and instruction-tuned open generative large language models (LLMs). The models are based on the GPT-3 decoder-only architecture and are pretrained on a mixture of Arabic and English texts, including source code in various programming languages. With 13 billion parameters, they demonstrate better knowledge and reasoning capabilities in Arabic than any existing open Arabic and multilingual models by a sizable margin, based on extensive evaluation. Moreover, the models are competitive in English compared to English-centric open models of similar size, despite being trained on much less English data. We provide a detailed description of the training, the tuning, the safety alignment, and the evaluation of the models. We release two open versions of the model -- the foundation Jais model, and an instruction-tuned Jais-chat variant -- with the aim of promoting research on Arabic LLMs. Available at https://huggingface.co/inception-mbzuai/jais-13b-chat

Key facts. Cited by 68 papers in the map (data through 7 Oct 2026). Builds on 79 of its 119 references that are in the map. First posted on arXiv 30 Aug 2023.

Builds on

79 of 119 references.

References · 79

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2023GPT-4 Technical Reportcited by 18,132
  5. 2022Training language models to follow instructions with human feedbackcited by 15,712
  6. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  7. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  8. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  9. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  10. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  11. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  12. 2020Scaling Laws for Neural Language Modelscited by 6,411
38 more
  1. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  2. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  3. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  4. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  5. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  6. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  7. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  8. 2021Program Synthesis with Large Language Modelscited by 3,509
  9. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  10. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  11. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  12. 2022Training Compute-Optimal Large Language Modelscited by 2,927
  13. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  14. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  15. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,818
  16. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  17. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,346
  18. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,140
  19. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,114
  20. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,968
  21. 2022Large Language Models Encode Clinical Knowledgecited by 1,807
  22. 2020GLU Variants Improve Transformercited by 1,724
  23. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,486
  24. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,462
  25. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,199
  26. 2019Cross-lingual Language Model Pretrainingcited by 1,088
  27. 2021Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolationcited by 1,085
  28. 2018Model Cards for Model Reportingcited by 1,068
  29. 2015Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Bookscited by 1,041
  30. 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,020
  31. 2021Ethical and social risks of harm from Language Modelscited by 957
  32. 2022Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Taskscited by 858
  33. 2017RACE: Large-scale ReAding Comprehension Dataset From Examinationscited by 851
  34. 2022GLM-130B: An Open Bilingual Pre-trained Modelcited by 837
  35. 2023The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Onlycited by 755
  36. 2020CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Modelscited by 703
  37. 2019Compressive Transformers for Long-Range Sequence Modellingcited by 699
  38. 2023Instruction Tuning with GPT-4cited by 676

50 most cited of 79 All 79 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 8; 2024: 20; 2025: 24; 2026: 16.

68 papers.

Sort:

By area:

  1. 2023Low-Resource Languages Jailbreak GPT-4cited by 290
  2. 2023Factuality Challenges in the Era of Large Language Modelscited by 125
  3. 2024Benchmarking Uncertainty Quantification Methods for Large Language Models with LM-Polygraphcited by 105
  4. 2024Survey of Cultural Awareness in Language Models: Text and Beyondcited by 101
  5. 2024LLMs know their vulnerabilities: Uncover Safety Gaps through Natural Distribution Shiftscited by 80
  6. 2023AceGPT, Localizing Large Language Models in Arabiccited by 75
  7. 2024ALLaM: Large Language Models for Arabic and Englishcited by 55
  8. 2025Power Lines: Scaling Laws for Weight Decay and Batch Size in LLM Pre-trainingcited by 41
  9. 2024Beyond English-Centric LLMs: What Language Do Multilingual Language Models Think in?cited by 40
  10. 2024IndoCulture: Exploring Geographically-Influenced Cultural Commonsense Reasoning Across Eleven Indonesian Provincescited by 38
  11. 2025Straight to Zero: Why Linearly Decaying the Learning Rate to Zero Works Best for LLMscited by 34
  12. 2023Efficiently Adapting Pretrained Language Models To New Languagescited by 28
38 more
  1. 2023Typhoon: Thai Large Language Modelscited by 28
  2. 2023Fine-tuning Large Language Models for Adaptive Machine Translationcited by 21
  3. 2023Domain Terminology Integration into Machine Translation: Leveraging Large Language Modelscited by 17
  4. 2024CAMEL-Bench: A Comprehensive Arabic LMM Benchmarkcited by 17
  5. 2025Sailor2: Sailing in South-East Asia with Inclusive Multilingual LLMscited by 17
  6. 2025Commonsense Reasoning in Arab Culturecited by 16
  7. 2024A Survey of Large Language Models for Arabic Language and its Dialectscited by 13
  8. 2024BenCzechMark : A Czech-centric Multitask and Multimetric Benchmark for Large Language Models with Duel Scoring Mechanismcited by 12
  9. 2025The Landscape of Arabic Large Language Models (ALLMs): A New Era for Arabic Language Technologycited by 12
  10. 2024SUTRA: Scalable Multilingual Language Model Architecturecited by 11
  11. 2024CamelEval: Advancing Culturally Aligned Arabic Language Models and Benchmarkscited by 10
  12. 2025Assessing Large Language Models on Islamic Legal Reasoning: Evidence from Inheritance Law Evaluationcited by 10
  13. 2024Native vs Non-Native Language Prompting: A Comparative Analysiscited by 9
  14. 2025SaudiCulture: A Benchmark for Evaluating Large Language Models Cultural Competence within Saudi Arabiacited by 8
  15. 2025Scaling with Collapse: Efficient and Predictable Training of LLM Familiescited by 8
  16. 2024ArabianGPT: Native Arabic GPT-based Large Language Modelcited by 7
  17. 2023Prediction of Arabic Legal Rulings using Large Language Modelscited by 6
  18. 2024Bilingual Adaptation of Monolingual Foundation Modelscited by 5
  19. 2025Hala Technical Report: Building Arabic-Centric Instruction & Translation Models at Scalecited by 5
  20. 2024Exploring Retrieval Augmented Generation in Arabiccited by 4
  21. 2025ARB: A Comprehensive Arabic Multimodal Reasoning Benchmarkcited by 4
  22. 2025QU-NLP at QIAS 2025 Shared Task: A Two-Phase LLM Fine-Tuning and Retrieval-Augmented Generation Approach for Islamic Inheritance Reasoningcited by 4
  23. 2024Language Modelling Approaches to Adaptive Machine Translationcited by 3
  24. 2025Not all tokens are created equal: Perplexity Attention Weighted Networks for AI generated text detectioncited by 3
  25. 2025Kuwain 1.5B: An Arabic SLM via Language Injectioncited by 3
  26. 2025The Arabic AI Fingerprint: Stylometric Analysis and Detection of Large Language Models Textcited by 3
  27. 2025K2-V2: A 360-Open, Reasoning-Enhanced LLMcited by 3
  28. 2026Toward Culturally Aligned LLMs through Ontology-Guided Multi-Agent Reasoningcited by 3
  29. 2024101 Billion Arabic Words Datasetcited by 2
  30. 2025AraTable: Benchmarking LLMs' Reasoning and Understanding of Arabic Tabular Datacited by 2
  31. 2025Saudi-Dialect-ALLaM: LoRA Fine-Tuning for Dialectal Arabic Generationcited by 2
  32. 2025Cross-Cultural Transfer of Commonsense Reasoning in LLMs: Evidence from the Arab Worldcited by 2
  33. 2025ALARB: An Arabic Legal Argument Reasoning Benchmarkcited by 2
  34. 2026SozKZ: Training Efficient Small Language Models for Kazakh from Scratchcited by 2
  35. 2026SAHM: A Benchmark for Arabic Financial and Shari'ah-Compliant Reasoningcited by 2
  36. 2026HalluScore: Large Language Model Hallucination Question Answering Benchmarkcited by 2
  37. 2024Desert Camels and Oil Sheikhs: Arab-Centric Red Teaming of Frontier LLMscited by 1
  38. 2024A Comparative Study of LLM-based ASR and Whisper in Low Resource and Code Switching Scenariocited by 1

50 most cited of 68 All 68 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025GemMaroc: Unlocking Darija Proficiency in LLMs with Minimal Data
  2. 2025Llama-3-Nanda-10B-Chat: An Open Generative Large Language Model for Hindi
  3. 2025Command R7B Arabic: A Small, Enterprise Focused, Multilingual, and Culturally Aware Arabic LLM
  4. 2025UI-Level Evaluation of ALLaM 34B: Measuring an Arabic-Centric LLM via HUMAIN Chat
  5. 2025Nile-Chat: Egyptian Language Models for Arabic and Latin Scripts
  6. 2025Tool Calling for Arabic LLMs: Data Strategies and Instruction Tuning
  7. 2023Chat Vector: A Simple Approach to Equip LLMs with Instruction Following and Model Alignment in New Languages
  8. 2025Mind the Gap: A Review of Arabic Post-Training Datasets and Their Limitations