mlsanityData to 14 Oct 2026

Building a Llama2-finetuned LLM for Odia Language Utilizing Domain Knowledge Instruction Set

arXiv 2312.12624 ↗First posted 19 Dec 2023cs.CLcited by 12

Not on the map. Why?

Abstract (arXiv)

Building LLMs for languages other than English is in great demand due to the unavailability and performance of multilingual LLMs, such as understanding the local context. The problem is critical for low-resource languages due to the need for instruction sets. In a multilingual country like India, there is a need for LLMs supporting Indic languages to provide generative AI and LLM-based technologies and services to its citizens. This paper presents our approach of i) generating a large Odia instruction set, including domain knowledge data suitable for LLM fine-tuning, and ii) building a Llama2-finetuned model tailored for enhanced performance in the Odia domain. The proposed work will help researchers build an instruction set and LLM, particularly for Indic languages. We will release the model and instruction set for the public for research and noncommercial purposes.

Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Builds on 14 of its 26 references that are in the map. First posted on arXiv 19 Dec 2023.

Builds on

14 of 26 references.

References · 14

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  2. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
  3. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,984
  4. 2023Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizescited by 737
  5. 2023Can Large Language Models Be an Alternative to Human Evaluations?cited by 734
  6. 20218-bit Optimizers via Block-wise Quantizationcited by 403
  7. 2022Outlier Suppression: Pushing the Limit of Low-bit Transformer Language Modelscited by 181
  8. 2021Samanantar: The Largest Publicly Available Parallel Corpora Collection for 11 Indic Languagescited by 136
  9. 2021The Shape of Learning Curves: a Reviewcited by 70
  10. 2016Explaining Predictions of Non-Linear Classifiers in NLPcited by 37
  11. 2021BLEU, METEOR, BERTScore: Evaluation of Metrics Performance in Assessing Critical Translation Errors in Sentiment-oriented Textcited by 22
  12. 2023Simple LLM Prompting is State-of-the-Art for Robust and Multilingual Dialogue Evaluationcited by 19
2 more
  1. 2021How to decay your learning ratecited by 11
  2. 2022Universal Dependency Treebank for Odia Languagecited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 7; 2025: 4; 2026: 1.

12 papers.

  1. 2024Multilingual Large Language Model: A Survey of Resources, Taxonomy and Frontierscited by 51
  2. 2025Krutrim LLM: Multilingual Foundational Model for over a Billion Peoplecited by 10
  3. 2025Chitrarth: Bridging Vision and Language for a Billion Peoplecited by 10
  4. 2024Multilingual Arbitrage: Optimizing Data Pools to Accelerate Multilingual Progresscited by 7
  5. 2024From Bytes to Borsch: Fine-Tuning Gemma and Mistral for the Ukrainian Language Representationcited by 6
  6. 2024LlamaTurk: Adapting Open-Source Generative Large Language Models for Low-Resource Languagecited by 6
  7. 2025Chitranuvad: Adapting Multi-Lingual LLMs for Multimodal Translationcited by 6
  8. 2024From English-Centric to Effective Bilingual: LLMs with Custom Tokenizers for Underrepresented Languagescited by 5
  9. 2024Decoding the Diversity: A Review of the Indic AI Research Landscapecited by 4
  10. 2025BhashaKritika: Building Synthetic Pretraining Data at Scale for Indic Languagescited by 2
  11. 2024Towards a More Inclusive AI: Progress and Perspectives in Large Language Model Training for the S'ami Languagecited by 1
  12. 2026Chitrakshara: A Large Multilingual Multimodal Dataset for Indian languagescited by 0