mlsanityData to 14 Oct 2026

PersianMind: A Cross-Lingual Persian-English Large Language Model

arXiv 2401.06466 ↗First posted 12 Jan 2024cs.CL cs.AIcited by 7

Not on the map. Why?

Abstract (arXiv)

Large language models demonstrate remarkable proficiency in various linguistic tasks and have extensive knowledge across various domains. Although they perform best in English, their ability in other languages is notable too. In contrast, open-source models, such as LLaMa, are primarily trained on English datasets, resulting in poor performance in non-English languages. In this paper, we introduce PersianMind, an open-source bilingual large language model which demonstrates comparable performance to closed-source GPT-3.5-turbo in the Persian language. By expanding LLaMa2's vocabulary with 10,000 Persian tokens and training it on a dataset comprising nearly 2 billion Persian tokens, we show that our approach preserves the model's English knowledge and employs transfer learning to excel at transferring task knowledge from one language to another.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 37 of its 49 references that are in the map. First posted on arXiv 12 Jan 2024.

Builds on

37 of 49 references.

References · 37

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2023GPT-4 Technical Reportcited by 18,396
  4. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  5. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  6. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  7. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  8. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  9. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
  10. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,918
  11. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
  12. 2023A Survey of Large Language Modelscited by 2,576
25 more
  1. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,818
  2. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,799
  3. 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,312
  4. 2021BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-modelscited by 1,245
  5. 2023PaLM 2 Technical Reportcited by 1,162
  6. 2023The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Onlycited by 763
  7. 2023Visual ChatGPT: Talking, Drawing and Editing with Visual Foundation Modelscited by 668
  8. 2022MTEB: Massive Text Embedding Benchmarkcited by 640
  9. 2020Language-agnostic BERT Sentence Embeddingcited by 627
  10. 2020Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillationcited by 593
  11. 2023Orca: Progressive Learning from Complex Explanation Traces of GPT-4cited by 379
  12. 2023The Belebele Benchmark: a Parallel Reading Comprehension Dataset in 122 Language Variantscited by 311
  13. 2019WikiMatrix: Mining 135M Parallel Sentences in 1620 Language Pairs from Wikipediacited by 223
  14. 2021Training Neural Networks with Fixed Sparse Maskscited by 214
  15. 2023Scaling Down to Scale Up: A Guide to Parameter-Efficient Fine-Tuningcited by 173
  16. 2022AdaMix: Mixture-of-Adaptations for Parameter-efficient Model Tuningcited by 137
  17. 2023AnglE-optimized Text Embeddingscited by 99
  18. 2020ParsBERT: Transformer-based Model for Persian Language Understandingcited by 93
  19. 2023XLM-V: Overcoming the Vocabulary Bottleneck in Multilingual Masked Language Modelscited by 88
  20. 2022SparseAdapter: An Easy Approach for Improving the Parameter-Efficiency of Adapterscited by 83
  21. 2023Bactrian-X: Multilingual Replicable Instruction-Following Models with Low-Rank Adaptationcited by 81
  22. 2022Bitext Mining Using Distilled Sentence Representations for Low-Resource Languagescited by 48
  23. 2020ParsiNLU: A Suite of Language Understanding Challenges for Persiancited by 42
  24. 2022Efficient Fine-Tuning of BERT Models on the Edgecited by 29
  25. 2020Leveraging ParsBERT and Pretrained mT5 for Persian Abstractive Text Summarizationcited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 3; 2025: 2; 2026: 2.

7 papers.

  1. 2025Leveraging Online Data to Enhance Medical Knowledge in a Small Persian Language Modelcited by 4
  2. 2024Extending LLMs to New Languages: A Case Study of Llama and Persian Adaptationcited by 3
  3. 2024PsychoLex: Unveiling the Psychological Mind of Large Language Modelscited by 2
  4. 2026PARSE: An Open-Domain Reasoning Question Answering Benchmark for Persiancited by 1
  5. 2024PersianRAG: A Retrieval-Augmented Generation System for Persian Languagecited by 0
  6. 2025Persian-Phi: Efficient Cross-Lingual Adaptation of Compact LLMs via Curriculum Learningcited by 0
  7. 2026The Annotation Bottleneck in Persian Text NLP: Persian as an Annotation-Scarce Languagecited by 0