mlsanityData to 14 Oct 2026

AceGPT, Localizing Large Language Models in Arabic

arXiv 2309.12053 ↗First posted 21 Sep 2023cs.CLcited by 75

On the map: LLM alignment / RLHF / preference optimisation → Human values and AI feedback alignment

Abstract (arXiv)

This paper is devoted to the development of a localized Large Language Model (LLM) specifically for Arabic, a language imbued with unique cultural characteristics inadequately addressed by current mainstream models. Significant concerns emerge when addressing cultural sensitivity and local values. To address this, the paper proposes a comprehensive solution that includes further pre-training with Arabic texts, Supervised Fine-Tuning (SFT) utilizing native Arabic instructions, and GPT-4 responses in Arabic, alongside Reinforcement Learning with AI Feedback (RLAIF) employing a reward model attuned to local culture and values. The goal is to cultivate culturally cognizant and value-aligned Arabic LLMs capable of accommodating the diverse, application-specific needs of Arabic-speaking communities. Comprehensive evaluations reveal that the resulting model, dubbed `AceGPT', sets the state-of-the-art standard for open Arabic LLMs across various benchmarks. Codes, data, and models are in https://github.com/FreedomIntelligence/AceGPT.

Key facts. Cited by 75 papers in the map (data through 14 Oct 2026). Placed in Human values and AI feedback alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 25 of its 49 references that are in the map. First posted on arXiv 21 Sep 2023.

Builds on

25 of 49 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedback22 May
  2. 2020Learning to summarize from human feedback2 Sep

Other references · 23

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  3. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  4. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  5. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,724
  6. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
  7. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,125
  8. 2023Large Language Models are not Fair Evaluatorscited by 1,085
  9. 2018XNLI: Evaluating Cross-lingual Sentence Representationscited by 915
  10. 2022Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Taskscited by 868
  11. 2023OpenAssistant Conversations -- Democratizing Large Language Model Alignmentcited by 721
  12. 2023C-Eval: A Multi-Level Multi-Discipline Chinese Evaluation Suite for Foundation Modelscited by 708
11 more
  1. 2023Instruction Tuning with GPT-4cited by 686
  2. 2023RLAIF vs. RLHF: Scaling Reinforcement Learning from Human Feedback with AI Feedbackcited by 625
  3. 2021Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpuscited by 492
  4. 2022Challenges and Strategies in Cross-Cultural NLPcited by 218
  5. 2023HuatuoGPT, towards Taming Language Model to Be a Doctorcited by 214
  6. 2023Jais and Jais-chat: Arabic-Centric Foundation and Instruction-Tuned Open Generative Large Language Modelscited by 69
  7. 2020EXAMS: A Multi-Subject High School Examinations Dataset for Cross-Lingual and Multilingual Question Answeringcited by 60
  8. 2023Knowledge of cultural moral norms in large language modelscited by 51
  9. 2023Phoenix: Democratizing ChatGPT across Languagescited by 32
  10. 2023Sociocultural Norm Similarities and Differences via Situational Alignment and Explainable Textual Entailmentcited by 22
  11. 2023AraMUS: Pushing the Limits of Data and Model Scale for Arabic Natural Language Processingcited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 25; 2025: 36; 2026: 14.

75 papers.

Sort:

By area:

  1. 2024ArabicMMLU: Assessing Massive Multitask Language Understanding in Arabiccited by 88
  2. 2024ALLaM: Large Language Models for Arabic and Englishcited by 55
  3. 2024AraDiCE: Benchmarks for Dialectal and Cultural Capabilities in LLMscited by 46
  4. 2024Self-Pluralising Culture Alignment for Large Language Modelscited by 31
  5. 2024NativQA: Multilingual Culturally-Aligned Natural Query for LLMscited by 30
  6. 2025Multiple LLM Agents Debate for Equitable Cultural Alignmentcited by 21
  7. 2025PalmX 2025: The First Shared Task on Benchmarking LLMs on Arabic and Islamic Culturecited by 21
  8. 2024CIDAR: Culturally Relevant Instruction Dataset For Arabiccited by 17
  9. 2024Arabic Dataset for LLM Safeguard Evaluationcited by 17
  10. 2024AraTrust: An Evaluation of Trustworthiness for LLMs in Arabiccited by 16
  11. 2024Dallah: A Dialect-Aware Multimodal Large Language Model for Arabiccited by 16
  12. 2024Cultural Palette: Pluralising Culture Alignment via Multi-agent Palettecited by 16
38 more
  1. 2025Command R7B Arabic: A Small, Enterprise Focused, Multilingual, and Culturally Aware Arabic LLMcited by 16
  2. 2025Fann or Flop: A Multigenre, Multiera Benchmark for Arabic Poetry Understanding in LLMscited by 16
  3. 2025Palm: A Culturally Inclusive and Linguistically Diverse Dataset for Arabic LLMscited by 15
  4. 2024Atlas-Chat: Adapting Large Language Models for Low-Resource Moroccan Arabic Dialectcited by 13
  5. 2024A Survey of Large Language Models for Arabic Language and its Dialectscited by 13
  6. 2025NileChat: Towards Linguistically Diverse and Culturally Aware LLMs for Local Communitiescited by 13
  7. 2025The Landscape of Arabic Large Language Models (ALLMs): A New Era for Arabic Language Technologycited by 12
  8. 2024AL-QASIDA: Analyzing LLM Quality and Accuracy Systematically in Dialectal Arabiccited by 11
  9. 2025Assessing Large Language Models on Islamic Legal Reasoning: Evidence from Inheritance Law Evaluationcited by 10
  10. 2024MQA-KEAL: Multi-hop Question Answering under Knowledge Editing for Arabic Languagecited by 9
  11. 2025Evaluating Arabic Large Language Models: A Survey of Benchmarks, Methods, and Gapscited by 9
  12. 2025PerCul: A Story-Driven Cultural Evaluation of LLMs in Persiancited by 8
  13. 2025SaudiCulture: A Benchmark for Evaluating Large Language Models Cultural Competence within Saudi Arabiacited by 8
  14. 2024ArabianGPT: Native Arabic GPT-based Large Language Modelcited by 7
  15. 2025Nile-Chat: Egyptian Language Models for Arabic and Latin Scriptscited by 7
  16. 2026IslamicMMLU: A Benchmark for Evaluating LLMs on Islamic Knowledgecited by 7
  17. 2025How well can LLMs Grade Essays in Arabic?cited by 6
  18. 2025LLM Alignment for the Arabs: A Homogenous Culture or Diverse Ones?cited by 6
  19. 2024Improving Language Models Trained on Translated Data with Continual Pre-Training and Dictionary Learning Analysiscited by 5
  20. 2025Cultural Alignment in Large Language Models Using Soft Prompt Tuningcited by 5
  21. 2025MOLE: Metadata Extraction and Validation in Scientific Papers Using LLMscited by 5
  22. 20253LM: Bridging Arabic, STEM, and Code through Benchmarkingcited by 5
  23. 2025BALSAM: A Platform for Benchmarking Arabic Large Language Modelscited by 5
  24. 2025Hala Technical Report: Building Arabic-Centric Instruction & Translation Models at Scalecited by 5
  25. 2025Document Intelligence in the Era of Large Language Models: A Surveycited by 5
  26. 2025FanarGuard: A Culturally-Aware Moderation Filter for Arabic Language Modelscited by 5
  27. 2024uDistil-Whisper: Label-Free Data Filtering for Knowledge Distillation in Low-Data Regimescited by 4
  28. 2024Second Language (Arabic) Acquisition of LLMs via Progressive Vocabulary Expansioncited by 4
  29. 2025Evaluating the Robustness and Accuracy of Text Watermarking Under Real-World Cross-Lingual Manipulationscited by 4
  30. 2024ArzEn-LLM: Code-Switched Egyptian Arabic-English Translation and Speech Recognition Using LLMscited by 3
  31. 2025Kuwain 1.5B: An Arabic SLM via Language Injectioncited by 3
  32. 2025Fine-Tuning LLMs for Low-Resource Dialect Translation: The Case of Lebanesecited by 3
  33. 2025The Arabic AI Fingerprint: Stylometric Analysis and Detection of Large Language Models Textcited by 3
  34. 2024GemmAr: Enhancing LLMs Through Arabic Instruction-Tuningcited by 2
  35. 2024ArabicNLU 2024: The First Arabic Natural Language Understanding Shared Taskcited by 2
  36. 2025Mutarjim: Advancing Bidirectional Arabic-English Translation with a Small Language Modelcited by 2
  37. 2025Do Large Language Models Know Folktales? A Case Study of Yokai in Japanese Folktalescited by 2
  38. 2025A Game-Theoretic Negotiation Framework for Cross-Cultural Consensus in LLMscited by 2

50 most cited of 75 All 75 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024CamelEval: Advancing Culturally Aligned Arabic Language Models and Benchmarks
  2. 2026State-of-the-Art Arabic Language Modeling with Sparse MoE Fine-Tuning and Chain-of-Thought Distillation
  3. 2024Alignment at Pre-training! Towards Native Alignment for Arabic LLMs
  4. 2025Cross-Cultural Transfer of Commonsense Reasoning in LLMs: Evidence from the Arab World
  5. 2026Cultural Benchmarking of LLMs in Standard and Dialectal Arabic Dialogues
  6. 2024The Qiyas Benchmark: Measuring ChatGPT Mathematical and Language Understanding in Arabic
  7. 2025Tool Calling for Arabic LLMs: Data Strategies and Instruction Tuning
  8. 2023GPT-Fathom: Benchmarking Large Language Models to Decipher the Evolutionary Path towards GPT-4 and Beyond