mlsanityData to 14 Oct 2026

Tamil-Llama: A New Tamil Language Model Based on Llama 2

arXiv 2311.05845 ↗First posted 10 Nov 2023cs.CL cs.AI cs.LGcited by 32

Not on the map. Why?

Abstract (arXiv)

Language modeling has witnessed remarkable advancements in recent years, with Large Language Models (LLMs) like ChatGPT setting unparalleled benchmarks in human-like text generation. However, a prevailing limitation is the underrepresentation of languages like Tamil in these cutting-edge models, leading to suboptimal performance in diverse linguistic contexts. This paper addresses this lacuna, enhancing the open-source LLaMA model with an addition of 16,000 Tamil tokens, aiming to achieve superior text generation and comprehension in the Tamil language. We strategically employ the LoRA methodology for efficient model training on a comprehensive Tamil corpus, ensuring computational feasibility and model robustness. Moreover, we introduce a Tamil-translated version of the Alpaca dataset and a subset of the OpenOrca dataset tailored for instruction fine-tuning. Our results showcase significant performance improvements in Tamil text generation, with potential implications for the broader landscape of LLMs in Indian languages. We further underscore our commitment to open research by making our models, datasets, and code publicly accessible, fostering further innovations in language modeling.

Key facts. Cited by 32 papers in the map (data through 14 Oct 2026). Builds on 8 of its 16 references that are in the map. First posted on arXiv 10 Nov 2023.

Builds on

8 of 16 references.

References · 8

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  4. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,989
  5. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,984
  6. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,346
  7. 2020Language ID in the Wild: Unexpected Challenges on the Path to a Thousand-Language Web Text Corpuscited by 77
  8. 2020AI4Bharat-IndicNLP Corpus: Monolingual Corpora and Word Embeddings for Indic Languagescited by 44

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 15; 2025: 10; 2026: 6.

32 papers.

Sort:

By area:

  1. 2024IndicGenBench: A Multilingual Benchmark to Evaluate Generation Capabilities of LLMs on Indic Languagescited by 56
  2. 2024PARIKSHA: A Large-Scale Investigation of Human-LLM Evaluator Agreement on Multilingual and Multi-Cultural Datacited by 37
  3. 2024Adapting Multilingual LLMs to Low-Resource Languages using Continued Pre-training and Synthetic Corpuscited by 24
  4. 2024How Can We Effectively Expand the Vocabulary of LLMs with 0.01GB of Target Language Text?cited by 22
  5. 2024Egalitarian Language Representation in Language Models: It All Begins with Tokenizerscited by 20
  6. 2024An Empirical Study on Cross-lingual Vocabulary Adaptation for Efficient Language Model Inferencecited by 18
  7. 2023PersianLLaMA: Towards Building First Persian Large Language Modelcited by 14
  8. 2025IndicMMLU-Pro: Benchmarking Indic Large Language Models on Multi-Task Language Understandingcited by 10
  9. 2025Krutrim LLM: Multilingual Foundational Model for over a Billion Peoplecited by 10
  10. 2024Efficient Continual Pre-training of LLMs for Low-resource Languagescited by 8
  11. 2024Adapting Chat Language Models Using Only Target Unlabeled Language Datacited by 6
  12. 2025Chitranuvad: Adapting Multi-Lingual LLMs for Multimodal Translationcited by 6
20 more
  1. 2025It's All About In-Context Learning! Teaching Extremely Low-Resource Languages to LLMscited by 6
  2. 2024Decoding the Diversity: A Review of the Indic AI Research Landscapecited by 4
  3. 2024Breaking Boundaries: Investigating the Effects of Model Editing on Cross-linguistic Performancecited by 4
  4. 2025Achieving Tokenizer Flexibility in Language Models through Heuristic Adaptation and Supertoken Learningcited by 4
  5. 2024[WIP] Jailbreak Paradox: The Achilles' Heel of LLMscited by 2
  6. 2024Krutrim LLM: A Novel Tokenization Strategy for Multilingual Indic Languages with Petabyte-Scale Data Processingcited by 2
  7. 2024Exploring Pretraining via Active Forgetting for Improving Cross Lingual Transfer for Decoder Language Modelscited by 2
  8. 2025Semantic Aware Linear Transfer by Recycling Pre-trained Language Models for Cross-lingual Transfercited by 2
  9. 2025BhashaKritika: Building Synthetic Pretraining Data at Scale for Indic Languagescited by 2
  10. 2024Suvach -- Generated Hindi QA benchmarkcited by 1
  11. 2024sPhinX: Sample Efficient Multilingual Instruction Fine-Tuning Through N-shot Guided Promptingcited by 1
  12. 2025UrduLLaMA 1.0: Dataset Curation, Preprocessing, and Evaluation in Low-Resource Settingscited by 1
  13. 2025Improving Multilingual Capabilities with Cultural and Local Knowledge in Large Language Models While Enhancing Native Performancecited by 1
  14. 2025Vocabulary Customization for Efficient Domain-Specific LLM Deploymentcited by 1
  15. 2026Large Language Models for Math Education in Low-Resource Languages: A Study in Sinhala and Tamilcited by 0
  16. 2026LGSE: Lexically Grounded Subword Embedding Initialization for Low-Resource Language Adaptationcited by 0
  17. 2026Indi-RomCoM: Code-Mixed Benchmark for Evaluating LLMs on Romanized Indic-English Instructionscited by 0
  18. 2026Rethinking Indic AI from a Lens of Cultural Heritage Preservationcited by 0
  19. 2026Contextual Tamil Spelling and Grammar Correction Using Progressively Fine-Tuned Sequence-to-Sequence Transformerscited by 0
  20. 2026Agaram 2.5B: The First Tamil-First Language Model Trained from Scratch with Tool Calling. A Playbook for Under-Represented Languagescited by 0