mlsanityData to 14 Oct 2026

CorIL: Towards Enriching Indian Language to Indian Language Parallel Corpora and Machine Translation Systems

arXiv 2509.19941 ↗First posted 24 Sep 2025cs.CL cs.AIcited by 1

Not on the map. Why?

Abstract (arXiv)

India's linguistic landscape is one of the most diverse in the world, comprising over 120 major languages and approximately 1,600 additional languages, with 22 officially recognized as scheduled languages in the Indian Constitution. Despite recent progress in multilingual neural machine translation (NMT), high-quality parallel corpora for Indian languages remain scarce, especially across varied domains. In this paper, we introduce a large-scale, high-quality annotated parallel corpus covering 11 of these languages : English, Telugu, Hindi, Punjabi, Odia, Kashmiri, Sindhi, Dogri, Kannada, Urdu, and Gujarati comprising a total of 772,000 bi-text sentence pairs. The dataset is carefully curated and systematically categorized into three key domains: Government, Health, and General, to enable domain-aware machine translation research and facilitate effective domain adaptation. To demonstrate the utility of CorIL and establish strong benchmarks for future research, we fine-tune and evaluate several state-of-the-art NMT models, including IndicTrans2, NLLB, and BhashaVerse. Our analysis reveals important performance trends and highlights the corpus's value in probing model capabilities. For instance, the results show distinct performance patterns based on language script, with massively multilingual models showing an advantage on Perso-Arabic scripts (Urdu, Sindhi) while other models excel on Indic scripts. This paper provides a detailed domain-wise performance analysis, offering insights into domain sensitivity and cross-script transfer learning. By publicly releasing CorIL, we aim to significantly improve the availability of high-quality training data for Indian languages and provide a valuable resource for the machine translation research community.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 11 of its 23 references that are in the map. First posted on arXiv 24 Sep 2025.

Builds on

11 of 23 references.

References · 11

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  2. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  3. 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,312
  4. 2019Cross-lingual Language Model Pretrainingcited by 1,090
  5. 2020COMET: A Neural Framework for MT Evaluationcited by 1,033
  6. 2023IndicTrans2: Towards High-Quality and Accessible Machine Translation Models for all 22 Scheduled Indian Languagescited by 152
  7. 2021Samanantar: The Largest Publicly Available Parallel Corpora Collection for 11 Indic Languagescited by 136
  8. 2017The IIT Bombay English-Hindi Parallel Corpuscited by 75
  9. 2019Translationese in Machine Translation Evaluationcited by 38
  10. 2020A Multilingual Parallel Corpora Collection Effort for Indian Languagescited by 16
  11. 2024BhashaVerse : Translation Ecosystem for Indian Subcontinent Languagescited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.