mlsanityData to 7 Oct 2026

PersianLLaMA: Towards Building First Persian Large Language Model

arXiv 2312.15713 ↗First posted 25 Dec 2023cs.CL cs.AIcited by 14

Not on the map. Why?

Abstract (arXiv)

Despite the widespread use of the Persian language by millions globally, limited efforts have been made in natural language processing for this language. The use of large language models as effective tools in various natural language processing tasks typically requires extensive textual data and robust hardware resources. Consequently, the scarcity of Persian textual data and the unavailability of powerful hardware resources have hindered the development of large language models for Persian. This paper introduces the first large Persian language model, named PersianLLaMA, trained on a collection of Persian texts and datasets. This foundational model comes in two versions, with 7 and 13 billion parameters, trained on formal and colloquial Persian texts using two different approaches. PersianLLaMA has been evaluated for natural language generation tasks based on the latest evaluation methods, namely using larger language models, and for natural language understanding tasks based on automated machine metrics. The results indicate that PersianLLaMA significantly outperforms its competitors in both understanding and generating Persian text. PersianLLaMA marks an important step in the development of Persian natural language processing and can be a valuable resource for the Persian-speaking community. This large language model can be used for various natural language processing tasks, especially text generation like chatbots, question-answering, machine translation, and text summarization

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 29 of its 35 references that are in the map. First posted on arXiv 25 Dec 2023.

Builds on

29 of 35 references.

References · 29

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  5. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  6. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  7. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  8. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  9. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  10. 2018Deep contextualized word representationscited by 2,285
  11. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
  12. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,813
17 more
  1. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,116
  2. 2023The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Onlycited by 755
  3. 2023OpenAssistant Conversations -- Democratizing Large Language Model Alignmentcited by 712
  4. 2023Orca: Progressive Learning from Complex Explanation Traces of GPT-4cited by 369
  5. 2023UniMax: Fairer and more Effective Language Sampling for Large-Scale Multilingual Pretrainingcited by 147
  6. 2022mGPT: Few-Shot Learners Go Multilingualcited by 141
  7. 2020A Monolingual Approach to Contextualized Word Embeddings for Mid-Resource Languagescited by 131
  8. 2019Neural Machine Translation with Byte-Level Subwordscited by 115
  9. 2023SeaLLMs -- Large Language Models for Southeast Asiacited by 104
  10. 2019Understanding LSTM -- a tutorial into Long Short-Term Memory Recurrent Neural Networkscited by 103
  11. 2023Tamil-Llama: A New Tamil Language Model Based on Llama 2cited by 31
  12. 2023Taiwan LLM: Bridging the Linguistic Divide with a Culturally Aligned Language Modelcited by 29
  13. 2022TencentPretrain: A Scalable and Flexible Toolkit for Pre-training Models of Different Modalitiescited by 19
  14. 2021SINA-BERT: A pre-trained Language Model for Analysis of Medical Texts in Persiancited by 10
  15. 2021MeDiaQA: A Question Answering Dataset on Medical Dialoguescited by 9
  16. 2022ArmanEmo: A Persian Dataset for Text-based Emotion Detectioncited by 9
  17. 2021PerCQA: Persian Community Question Answering Datasetcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 8; 2025: 4; 2026: 2.

14 papers.

Sort:

By area:

  1. 2024Survey of Cultural Awareness in Language Models: Text and Beyondcited by 101
  2. 2024All Languages Matter: Evaluating LMMs on Culturally Diverse 100 Languagescited by 55
  3. 2024CultureLLM: Incorporating Cultural Differences into Large Language Modelscited by 49
  4. 2024CulturePark: Boosting Cross-cultural Understanding in Large Language Modelscited by 24
  5. 2024Sabi'a-2: A New Generation of Portuguese Large Language Modelscited by 10
  6. 2025CAReDiO: Cultural Alignment via Representativeness and Distinctiveness Guided Data Optimizationcited by 5
  7. 2024PsychoLex: Unveiling the Psychological Mind of Large Language Modelscited by 2
  8. 2025Advancing Retrieval-Augmented Generation for Persian: Development of Language Models, Comprehensive Benchmarks, and Best Practices for Optimizationcited by 2
  9. 2025Towards culturally-appropriate conversational AI for health in the majority world: An exploratory study with citizens and professionals in Latin Americacited by 1
  10. 2025Benchmarking Open-Source Large Language Models for Persian in Zero-Shot and Few-Shot Learningcited by 1
  11. 2026PARSE: An Open-Domain Reasoning Question Answering Benchmark for Persiancited by 1
  12. 2024PerkwE_COQA: Enhanced Persian Conversational Question Answering by combining contextual keyword extraction with Large Language Modelscited by 0
2 more
  1. 2024PersianRAG: A Retrieval-Augmented Generation System for Persian Languagecited by 0
  2. 2026Structure Matters: Evaluating Multi-Agents Orchestration in Generative Therapeutic Chatbotscited by 0