GemMaroc: Unlocking Darija Proficiency in LLMs with Minimal Data
On the map: LLM reasoning → Reasoning benchmarks and evaluation
Abstract (arXiv)
Open-source large language models (LLMs) still marginalise Moroccan Arabic (Darija), forcing practitioners either to bolt on heavyweight Arabic adapters or to sacrifice the very reasoning skills that make LLMs useful. We show that a rigorously quality-over-quantity alignment strategy can surface fluent Darija while safeguarding the backbone s cross-lingual reasoning at a sliver of the usual compute. We translate three compact instruction suites LIMA 1 K, DEITA 6 K and TULU 50 K into Darija, preserve 20 of the English originals, and add mathematics, coding and scientific prompts. A LoRA-tuned Gemma 3-4B trained on 5 K mixed instructions lifts DarijaMMLU from 32.8 to 42.7 ; adding the reasoning-dense TULU portion pushes it to 47.5 with no English regression. Scaling the identical recipe to Gemma 3-27B produces GemMaroc-27B, which matches Atlas-Chat on DarijaMMLU (61.6 ) and leaps ahead on Darija commonsense, scoring 60.5 on HellaSwag versus Atlas-Chat s 48.4 . Crucially, GemMaroc retains Gemma-27B s strong maths and general-reasoning ability, showing only minimal movement on GSM8K and English benchmarks. The entire model is trained in just 48 GPU.h, underscoring a Green AI pathway to inclusive, sustainable language technology. We release code, data and checkpoints to spur Darija-centric applications in education, public services and everyday digital interaction.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 21 of its 36 references that are in the map. First posted on arXiv 20 May 2025.
Builds on
21 of 36 references.
Closest ancestors · 1
Same direction, newest first
Other references · 20
- 2021Training Verifiers to Solve Math Word Problemscited by 10,109
- 2020Measuring Massive Multitask Language Understandingcited by 7,963
- 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,100
- 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
- 2023LIMA: Less Is More for Alignmentcited by 1,100
- 2019Quantifying the Carbon Emissions of Machine Learningcited by 521
- 2023What Makes Good Data for Alignment? A Comprehensive Study of Automatic Data Selection in Instruction Tuningcited by 364
- 2020Towards the Systematic Reporting of the Energy and Carbon Footprints of Machine Learningcited by 249
- 2025Fanar: An Arabic-Centric Multimodal Generative AI Platformcited by 67
- 2024ALLaM: Large Language Models for Arabic and Englishcited by 55
- 2024X-Instruction: Aligning Language Model in Low-resource Languages with Self-curated Cross-lingual Instructionscited by 18
- 2024Mixture-of-Skills: Learning to Optimize Data Usage for Fine-Tuning Large Language Modelscited by 16
8 more
- 2024Atlas-Chat: Adapting Large Language Models for Low-Resource Moroccan Arabic Dialectcited by 13
- 2024MURI: High-Quality Instruction Tuning Datasets for Low-Resource Languages via Reverse Instructionscited by 12
- 2025MIG: Automatic Data Selection for Instruction Tuning by Maximizing Information Gain in Semantic Spacecited by 12
- 2024Investigating Multilingual Instruction-Tuning: Do Polyglot Models Demand for Multilingual Instructions?cited by 10
- 2024LIONs: An Empirically Optimized Approach to Align Language Modelscited by 7
- 2023DEFT: Data Efficient Fine-Tuning for Pre-Trained Language Models via Unsupervised Core-Set Selectioncited by 6
- 2024Boosting LLM Translation Skills without General Ability Loss via Rationale Distillationcited by 6
- 2025Fine-Tuning LLMs for Low-Resource Dialect Translation: The Case of Lebanesecited by 3
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2024Alignment at Pre-training! Towards Native Alignment for Arabic LLMs
- 2024CamelEval: Advancing Culturally Aligned Arabic Language Models and Benchmarks
- 2023Jais and Jais-chat: Arabic-Centric Foundation and Instruction-Tuned Open Generative Large Language Models
- 2025Tool Calling for Arabic LLMs: Data Strategies and Instruction Tuning
- 2026State-of-the-Art Arabic Language Modeling with Sparse MoE Fine-Tuning and Chain-of-Thought Distillation
- 2025UI-Level Evaluation of ALLaM 34B: Measuring an Arabic-Centric LLM via HUMAIN Chat
- 2025AraReasoner: Evaluating Reasoning-Based LLMs for Arabic NLP
- 2025DialectalArabicMMLU: Benchmarking Dialectal Capabilities in Arabic and Multilingual Language Models