mlsanityData to 14 Oct 2026

SinLlama -- A Large Language Model for Sinhala

arXiv 2508.09115 ↗First posted 12 Aug 2025cs.CLcited by 9

Not on the map. Why?

Abstract (arXiv)

Low-resource languages such as Sinhala are often overlooked by open-source Large Language Models (LLMs). In this research, we extend an existing multilingual LLM (Llama-3-8B) to better serve Sinhala. We enhance the LLM tokenizer with Sinhala specific vocabulary and perform continual pre-training on a cleaned 10 million Sinhala corpus, resulting in the SinLlama model. This is the very first decoder-based open-source LLM with explicit Sinhala support. When SinLlama was instruction fine-tuned for three text classification tasks, it outperformed base and instruct variants of Llama-3-8B by a significant margin.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Builds on 37 of its 48 references that are in the map. First posted on arXiv 12 Aug 2025.

Builds on

37 of 48 references.

References · 37

  1. 2017Attention Is All You Needcited by 60,361
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  5. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  6. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,899
  7. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  8. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  9. 2023Mistral 7Bcited by 3,164
  10. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
  11. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,917
  12. 2023A Survey on Evaluation of Large Language Modelscited by 1,242
25 more
  1. 2024Gemma: Open Models Based on Gemini Research and Technologycited by 916
  2. 2019DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphscited by 903
  3. 2020The State and Fate of Linguistic Diversity and Inclusion in the NLP Worldcited by 798
  4. 2023The Falcon Series of Open Language Modelscited by 535
  5. 2023Efficient and Effective Text Encoding for Chinese LLaMA and Alpacacited by 305
  6. 2024Aya Model: An Instruction Finetuned Open-Access Multilingual Language Modelcited by 295
  7. 2023MEGA: Multilingual Evaluation of Generative AIcited by 293
  8. 2023MADLAD-400: A Multilingual And Document-Level Large Audited Datasetcited by 181
  9. 2023CulturaX: A Cleaned, Enormous, and Multilingual Dataset for Large Language Models in 167 Languagescited by 174
  10. 2023Don't Trust ChatGPT when Your Question is not in English: A Study of Multilingual Abilities and Types of LLMscited by 104
  11. 2022Continual Pre-Training Mitigates Forgetting in Language and Visioncited by 83
  12. 2023PolyLM: An Open Source Polyglot Large Language Modelcited by 64
  13. 2023BigTranslate: Augmenting Large Language Models with Multilingual Translation Capability over 100 Languagescited by 60
  14. 2023Monolingual or Multilingual Instruction Tuning: Which Makes a Better Alpacacited by 60
  15. 2023Adaptive Machine Translation with Large Language Modelscited by 58
  16. 2023Eliciting the Translation Ability of Large Language Models via Multilingual Finetuning with Translation Instructionscited by 56
  17. 2022Some Languages are More Equal than Others: Probing Deeper into the Linguistic Disparity in the NLP Worldcited by 55
  18. 2019Survey on Publicly Available Sinhala Natural Language Processing Tools and Researchcited by 46
  19. 2024MaLA-500: Massive Language Adaptation of Large Language Modelscited by 28
  20. 2023TaCo: Enhancing Cross-Lingual Transfer for Low-Resource Languages in LLMs through Translation-Assisted Chain-of-Thought Processescited by 19
  21. 2022BERTifying Sinhala -- A Comprehensive Analysis of Pre-trained Language Models for Sinhala Text Classificationcited by 16
  22. 2024Zero-shot Sentiment Analysis in Low-Resource Languages Using a Multilingual Sentiment Lexiconcited by 7
  23. 2024LlamaTurk: Adapting Open-Source Generative Large Language Models for Low-Resource Languagecited by 6
  24. 2024Shoulders of Giants: A Look at the Degree and Utility of Openness in NLP Researchcited by 4
  25. 2025Improving the quality of Web-mined Parallel Corpora of Low-Resource Languages using Debiasing Heuristicscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 8.

9 papers.

  1. 2026Apples to Apples? Towards Comparable Crosslingual Language Model Evaluationcited by 3
  2. 2026SinhaLegal: A Benchmark Corpus for Information Extraction and Analysis in Sinhala Legislative Textscited by 1
  3. 2026Separate Before You Compress: The WWHO Tokenization Architecturecited by 1
  4. 2025LMSpell: Spell Correction with Pre-Trained Language Modelscited by 0
  5. 2026Sinhala Physical Common Sense Reasoning Dataset for Global PIQAcited by 0
  6. 2026SiPaKosa: A Comprehensive Corpus of Canonical and Classical Buddhist Texts in Sinhala and Palicited by 0
  7. 2026HelaBERT: Enhancing Sinhala Language Understanding with Dual Pooling Classification Headcited by 0
  8. 2026Dynamics of Meaning: Towards the Evaluation of Diachronic Semantic Change in Sinhalacited by 0
  9. 2026Automatic Speech Recognition for Low-Resource Sinhala: A Critical Review of Methods, Challenges, and Future Directionscited by 0