mlsanityData to 14 Oct 2026

Krutrim LLM: Multilingual Foundational Model for over a Billion People

arXiv 2502.09642 ↗First posted 10 Feb 2025cs.CL cs.AIcited by 10

Not on the map. Why?

Abstract (arXiv)

India is a diverse society with unique challenges in developing AI systems, including linguistic diversity, oral traditions, data accessibility, and scalability. Existing foundation models are primarily trained on English, limiting their effectiveness for India's population. Indic languages comprise only 1 percent of Common Crawl corpora despite India representing 18 percent of the global population, leading to linguistic biases. Thousands of regional languages, dialects, and code mixing create additional representation challenges due to sparse training data. We introduce Krutrim LLM, a 2 trillion token multilingual model designed for India's linguistic landscape. It incorporates the largest known Indic dataset, mitigating data scarcity and ensuring balanced performance across dialects. Krutrim outperforms or matches state-of-the-art models on Indic benchmarks while maintaining competitive English performance. Despite being significantly smaller in training flops, Krutrim LLM matches or exceeds models like LLAMA-2 on 10 out of 16 tasks, with an average score of 0.57 versus 0.55. This evidences Krutrim's flexible multilingual fluency across diverse linguistic contexts. Krutrim is integrated with real-time search to improve factual accuracy in conversational AI applications. This enhances accessibility for over 1 billion users worldwide. Through intentional design choices addressing data imbalances, Krutrim LLM signifies meaningful progress in building ethical, globally representative AI models.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Builds on 29 of its 38 references that are in the map. First posted on arXiv 10 Feb 2025.

Builds on

29 of 38 references.

References · 29

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2017Proximal Policy Optimization Algorithmscited by 15,215
  4. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  5. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  6. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  7. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  8. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  9. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
  10. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,898
  11. 2020Learning to summarize from human feedbackcited by 2,588
  12. 2019Language Models as Knowledge Bases?cited by 1,936
17 more
  1. 2022Language Models (Mostly) Know What They Knowcited by 1,856
  2. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,346
  3. 2021Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolationcited by 1,103
  4. 2021A General Language Assistant as a Laboratory for Alignmentcited by 1,055
  5. 2023The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Onlycited by 763
  6. 2024Dolma: an Open Corpus of Three Trillion Tokens for Language Model Pretraining Researchcited by 483
  7. 2020ReClor: A Reading Comprehension Dataset Requiring Logical Reasoningcited by 217
  8. 2023Continual Pre-training of Language Modelscited by 198
  9. 2023IndicTrans2: Towards High-Quality and Accessible Machine Translation Models for all 22 Scheduled Indian Languagescited by 152
  10. 2022Towards Leaving No Indic Language Behind: Building Monolingual Corpora, Benchmark and Models for Indic Languagescited by 102
  11. 2024Self-Distillation Bridges Distribution Gap in Language Model Fine-Tuningcited by 96
  12. 2022Continual Pre-Training Mitigates Forgetting in Language and Visioncited by 83
  13. 2024IndicLLMSuite: A Blueprint for Creating Pre-training and Fine-Tuning Datasets for Indian Languagescited by 52
  14. 2017Composite Task-Completion Dialogue Policy Learning via Hierarchical Deep Reinforcement Learningcited by 44
  15. 2023Tamil-Llama: A New Tamil Language Model Based on Llama 2cited by 32
  16. 2024Airavata: Introducing Hindi Instruction-tuned LLMcited by 32
  17. 2023Building a Llama2-finetuned LLM for Odia Language Utilizing Domain Knowledge Instruction Setcited by 12

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 4; 2026: 6.

10 papers.