mlsanityData to 14 Oct 2026

Hala Technical Report: Building Arabic-Centric Instruction & Translation Models at Scale

arXiv 2509.14008 ↗First posted 17 Sep 2025cs.CL cs.AI cs.LGcited by 5

Not on the map. Why?

Abstract (arXiv)

We present Hala, a family of Arabic-centric instruction and translation models built with our translate-and-tune pipeline. We first compress a strong ARleftrightarrowEN teacher to FP8 (yielding ~2× higher throughput with no quality loss) and use it to create high-fidelity bilingual supervision. A lightweight language model LFM2-1.2B is then fine-tuned on this data and used to translate high-quality English instruction sets into Arabic, producing a million-scale corpus tailored to instruction following. We train Hala models at 350M, 700M, 1.2B, and 9B parameters, and apply slerp merging to balance Arabic specialization with base-model strengths. On Arabic-centric benchmarks, Hala achieves state-of-the-art results within both the "nano" (≤2B) and "small" (7-9B) categories, outperforming their bases. We release models, data, evaluation, and recipes to accelerate research in Arabic NLP.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 41 of its 51 references that are in the map. First posted on arXiv 17 Sep 2025.

Builds on

41 of 51 references.

References · 41

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2024The Llama 3 Herd of Modelscited by 14,715
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  5. 2025Qwen3 Technical Reportcited by 8,820
  6. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  7. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
  8. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,724
  9. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,984
  10. 2025Gemma 3 Technical Reportcited by 1,824
  11. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,818
  12. 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Trainingcited by 1,053
29 more
  1. 2023Baichuan 2: Open Large-scale Language Modelscited by 670
  2. 2025Kimi K2: Open Agentic Intelligencecited by 567
  3. 2023The Falcon Series of Open Language Modelscited by 535
  4. 2023Orca: Progressive Learning from Complex Explanation Traces of GPT-4cited by 379
  5. 2020AraBERT: Transformer-based Model for Arabic Language Understandingcited by 362
  6. 2020Improving Massively Multilingual Neural Machine Translation and Zero-Shot Translationcited by 277
  7. 2024Arcee's MergeKit: A Toolkit for Merging Large Language Modelscited by 194
  8. 2023Multilingual Machine Translation with Large Language Models: Empirical Results and Analysiscited by 160
  9. 2022MASSIVE: A 1M-Example Multilingual Natural Language Understanding Dataset with 51 Typologically-Diverse Languagescited by 146
  10. 2020The Tatoeba Translation Challenge -- Realistic Data Sets for Low Resource and Multilingual MTcited by 128
  11. 2023LaMini-LM: A Diverse Herd of Distilled Models from Large-Scale Instructionscited by 124
  12. 2022FP8 Quantization: The Power of the Exponentcited by 88
  13. 2023AceGPT, Localizing Large Language Models in Arabiccited by 75
  14. 2023Jais and Jais-chat: Arabic-Centric Foundation and Instruction-Tuned Open Generative Large Language Modelscited by 69
  15. 2025Fanar: An Arabic-Centric Multimodal Generative AI Platformcited by 67
  16. 2020EXAMS: A Multi-Subject High School Examinations Dataset for Cross-Lingual and Multilingual Question Answeringcited by 60
  17. 2024A New Massive Multilingual Dataset for High-Performance Language Technologiescited by 56
  18. 2024ALLaM: Large Language Models for Arabic and Englishcited by 55
  19. 2023Exploring Human-Like Translation Strategy with Large Language Modelscited by 44
  20. 2023Early Weight Averaging meets High Learning Rates for LLM Pre-trainingcited by 37
  21. 2024Hermes 3 Technical Reportcited by 34
  22. 2024Peacock: A Family of Arabic Multimodal Large Language Models and Benchmarkscited by 32
  23. 2025SCP-116K: A High-Quality Problem-Solution Dataset and a Generalized Pipeline for Automated Extraction in the Higher Education Science Domaincited by 31
  24. 2024Beneath the Surface of Consistency: Exploring Cross-lingual Knowledge Representation Sharing in LLMscited by 21
  25. 2023A Paradigm Shift: The Future of Machine Translation Lies with Large Language Modelscited by 19
  26. 2024AraTrust: An Evaluation of Trustworthiness for LLMs in Arabiccited by 16
  27. 2024Reformatted Alignmentcited by 15
  28. 2025The Landscape of Arabic Large Language Models (ALLMs): A New Era for Arabic Language Technologycited by 12
  29. 2025Science Across Languages: Assessing LLM Multilingual Translation of Scientific Paperscited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 3.

5 papers.

  1. 2025AraLingBench A Human-Annotated Benchmark for Evaluating Arabic Linguistic Capabilities of Large Language Modelscited by 2
  2. 2026Polyglot Teachers: Evaluating Language Models for Multilingual Synthetic Data Generationcited by 1
  3. 2025SmolKalam: Ensemble Quality-Filtered Translation at Scale for High Quality Arabic Post-Training Datacited by 0
  4. 2026Cultural Benchmarking of LLMs in Standard and Dialectal Arabic Dialoguescited by 0
  5. 2026Arabic Safety Alignment as Selective Refusal: An Empirical Study of SFT, DPO, and Guard Calibrationcited by 0