Persian-Phi: Efficient Cross-Lingual Adaptation of Compact LLMs via Curriculum Learning
Not on the map. Why?
Abstract (arXiv)
The democratization of AI is currently hindered by the immense computational costs required to train Large Language Models (LLMs) for low-resource languages. This paper presents Persian-Phi, a 3.8B parameter model that challenges the assumption that robust multilingual capabilities require massive model sizes or multilingual baselines. We demonstrate how Microsoft Phi-3 Mini -- originally a monolingual English model -- can be effectively adapted to Persian through a novel, resource-efficient curriculum learning pipeline. Our approach employs a unique "warm-up" stage using bilingual narratives (Tiny Stories) to align embeddings prior to heavy training, followed by continual pretraining and instruction tuning via Parameter-Efficient Fine-Tuning (PEFT). Despite its compact size, Persian-Phi achieves competitive results on Open Persian LLM Leaderboard in HuggingFace. Our findings provide a validated, scalable framework for extending the reach of state-of-the-art LLMs to underrepresented languages with minimal hardware resources. The Persian-Phi model is publicly available at https://huggingface.co/amirakhlaghiqqq/PersianPhi.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 39 of its 52 references that are in the map. First posted on arXiv 8 Dec 2025.
Builds on
39 of 52 references.
References · 39
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2024The Llama 3 Herd of Modelscited by 14,715
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
- 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
- 2021Evaluating Large Language Models Trained on Codecited by 8,320
- 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
- 2021Program Synthesis with Large Language Modelscited by 3,637
- 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
- 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,918
- 2019The Curious Case of Neural Text Degenerationcited by 2,916
- 2022Emergent Abilities of Large Language Modelscited by 2,512
27 more
- 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,917
- 2024Mixtral of Expertscited by 1,823
- 2016Bag of Tricks for Efficient Text Classificationcited by 876
- 2021Deduplicating Training Data Makes Language Models Bettercited by 714
- 2024Phi-4 Technical Reportcited by 711
- 2020Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillationcited by 593
- 2023Textbooks Are All You Needcited by 569
- 2024Dolma: an Open Corpus of Three Trillion Tokens for Language Model Pretraining Researchcited by 483
- 2023TinyStories: How Small Can Language Models Be and Still Speak Coherent English?cited by 449
- 2024LongRoPE: Extending LLM Context Window Beyond 2 Million Tokenscited by 313
- 2023Efficient and Effective Text Encoding for Chinese LLaMA and Alpacacited by 305
- 2023A Pretrainer's Guide to Training Data: Measuring the Effects of Data Age, Domain Coverage, Quality, & Toxicitycited by 182
- 2024RedPajama: an Open Dataset for Training Large Language Modelscited by 182
- 2022Scaling Laws and Interpretability of Learning from Repeated Datacited by 152
- 2024LLaMA Beyond English: An Empirical Study on Language Capability Transfercited by 95
- 2023Bactrian-X: Multilingual Replicable Instruction-Following Models with Low-Rank Adaptationcited by 81
- 2021What's in the Box? A Preliminary Analysis of Undesirable Content in the Common Crawl Corpuscited by 80
- 2023Extrapolating Large Language Models to Non-English by Aligning Languagescited by 69
- 2023PLUG: Leveraging Pivot Language in Cross-Lingual Instruction Tuningcited by 39
- 2023When Is Multilinguality a Curse? Language Modeling for 250 High- and Low-Resource Languagescited by 32
- 2023Efficiently Adapting Pretrained Language Models To New Languagescited by 28
- 2022Mini-Model Adaptation: Efficiently Extending Pretrained Models to New Languages via Aligned Shallow Trainingcited by 24
- 2024Teaching Llama a New Language Through Cross-Lingual Knowledge Transfercited by 23
- 2024Teaching LLMs to Abstain across Languages via Multilingual Feedbackcited by 14
- 2024PersianMind: A Cross-Lingual Persian-English Large Language Modelcited by 7
- 2024InstructionCP: A fast approach to transfer Large Language Models into target languagecited by 5
- 2021ParsiNorm: A Persian Toolkit for Speech Processing Normalizationcited by 4
Built on by
Not cited yet.