# IndicNLG Benchmark: Multilingual Datasets for Diverse NLG Tasks in Indic Languages

arXiv: https://arxiv.org/abs/2203.05437 · HTML: https://mlsanity.com/paper/2203.05437/

## Key facts

- Cited by 42 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 65 references that are in the map.
- First posted on arXiv 10 Mar 2022.

## Builds on (most cited first, 30 of 34)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [mT5: A massively multilingual pre-trained text-to-text transformer](https://mlsanity.com/paper/2010.11934/) (2020)
- [A Call for Clarity in Reporting BLEU Scores](https://mlsanity.com/paper/1804.08771/) (2018)
- [Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarization](https://mlsanity.com/paper/1808.08745/) (2018)
- [Multilingual Denoising Pre-training for Neural Machine Translation](https://mlsanity.com/paper/2001.08210/) (2020)
- [SemEval-2017 Task 1: Semantic Textual Similarity - Multilingual and Cross-lingual Focused Evaluation](https://mlsanity.com/paper/1708.00055/) (2017)
- [Unified Language Model Pre-training for Natural Language Understanding and Generation](https://mlsanity.com/paper/1905.03197/) (2019)
- [On the Cross-lingual Transferability of Monolingual Representations](https://mlsanity.com/paper/1910.11856/) (2019)
- [TyDi QA: A Benchmark for Information-Seeking Question Answering in Typologically Diverse Languages](https://mlsanity.com/paper/2003.05002/) (2020)
- [A Neural Attention Model for Abstractive Sentence Summarization](https://mlsanity.com/paper/1509.00685/) (2015)
- [XL-Sum: Large-Scale Multilingual Abstractive Summarization for 44 Languages](https://mlsanity.com/paper/2106.13822/) (2021)
- [MLQA: Evaluating Cross-lingual Extractive Question Answering](https://mlsanity.com/paper/1910.07475/) (2019)
- [ToTTo: A Controlled Table-To-Text Generation Dataset](https://mlsanity.com/paper/2004.14373/) (2020)
- [The GEM Benchmark: Natural Language Generation, its Evaluation and Metrics](https://mlsanity.com/paper/2102.01672/) (2021)
- [Challenges in Data-to-Document Generation](https://mlsanity.com/paper/1707.08052/) (2017)
- [Survey of the State of the Art in Natural Language Generation: Core tasks, applications and evaluation](https://mlsanity.com/paper/1703.09902/) (2017)
- [Learning to Ask: Neural Question Generation for Reading Comprehension](https://mlsanity.com/paper/1705.00106/) (2017)
- [WikiLingua: A New Benchmark Dataset for Cross-Lingual Abstractive Summarization](https://mlsanity.com/paper/2010.03093/) (2020)
- [DART: Open-Domain Structured Data Record to Text Generation](https://mlsanity.com/paper/2007.02871/) (2020)
- [Samanantar: The Largest Publicly Available Parallel Corpora Collection for 11 Indic Languages](https://mlsanity.com/paper/2104.05596/) (2021)
- [MLSUM: The Multilingual Summarization Corpus](https://mlsanity.com/paper/2004.14900/) (2020)
- [Text-to-Text Pre-Training for Data-to-Text Tasks](https://mlsanity.com/paper/2005.10433/) (2020)
- [Evaluating the State-of-the-Art of End-to-End Natural Language Generation: The E2E NLG Challenge](https://mlsanity.com/paper/1901.07931/) (2019)
- [Neural Question Generation from Text: A Preliminary Study](https://mlsanity.com/paper/1704.01792/) (2017)
- [IndoNLG: Benchmark and Resources for Evaluating Indonesian Natural Language Generation](https://mlsanity.com/paper/2104.08200/) (2021)
- [IndicBART: A Pre-trained Model for Indic Natural Language Generation](https://mlsanity.com/paper/2109.02903/) (2021)
- [Data-to-text Generation with Macro Planning](https://mlsanity.com/paper/2102.02723/) (2021)

## Built on by (most cited first, 30 of 42)

- [LLMs Are Few-Shot In-Context Low-Resource Language Learners](https://mlsanity.com/paper/2403.16512/) (2024)
- [IndicGenBench: A Multilingual Benchmark to Evaluate Generation Capabilities of LLMs on Indic Languages](https://mlsanity.com/paper/2404.16816/) (2024)
- [SEACrowd: A Multilingual Multimodal Data Hub and Benchmark Suite for Southeast Asian Languages](https://mlsanity.com/paper/2406.10118/) (2024)
- [NusaCrowd: Open Source Initiative for Indonesian NLP Resources](https://mlsanity.com/paper/2212.09648/) (2022)
- [PARIKSHA: A Large-Scale Investigation of Human-LLM Evaluator Agreement on Multilingual and Multi-Cultural Data](https://mlsanity.com/paper/2406.15053/) (2024)
- [Airavata: Introducing Hindi Instruction-tuned LLM](https://mlsanity.com/paper/2401.15006/) (2024)
- [RomanSetu: Efficiently unlocking multilingual capabilities of Large Language Models via Romanization](https://mlsanity.com/paper/2401.14280/) (2024)
- [Naamapadam: A Large-Scale Named Entity Annotated Data for Indic Languages](https://mlsanity.com/paper/2212.10168/) (2022)
- [Adapting Multilingual LLMs to Low-Resource Languages using Continued Pre-training and Synthetic Corpus](https://mlsanity.com/paper/2410.14815/) (2024)
- [BenLLMEval: A Comprehensive Evaluation into the Potentials and Pitfalls of Large Language Models on Bengali NLP](https://mlsanity.com/paper/2309.13173/) (2023)
- [Dolphin: A Challenging and Diverse Benchmark for Arabic NLG](https://mlsanity.com/paper/2305.14989/) (2023)
- [InstructAlign: High-and-Low Resource Language Alignment via Continual Crosslingual Instruction Tuning](https://mlsanity.com/paper/2305.13627/) (2023)
- [Cheetah: Natural Language Generation for 517 African Languages](https://mlsanity.com/paper/2401.01053/) (2024)
- [mEdIT: Multilingual Text Editing via Instruction Tuning](https://mlsanity.com/paper/2402.16472/) (2024)
- [One Tokenizer To Rule Them All: Emergent Language Plasticity via Multilingual Tokenizers](https://mlsanity.com/paper/2506.10766/) (2025)
- [PMIndiaSum: Multilingual and Cross-lingual Headline Summarization for Languages in India](https://mlsanity.com/paper/2305.08828/) (2023)
- [Evaluating Tokenizer Performance of Large Language Models Across Official Indian Languages](https://mlsanity.com/paper/2411.12240/) (2024)
- [IndicMMLU-Pro: Benchmarking Indic Large Language Models on Multi-Task Language Understanding](https://mlsanity.com/paper/2501.15747/) (2025)
- [Llama-3-Nanda-10B-Chat: An Open Generative Large Language Model for Hindi](https://mlsanity.com/paper/2504.06011/) (2025)
- [Rethinking Tokenization for Rich Morphology: The Dominance of Unigram over BPE and Morphological Alignment](https://mlsanity.com/paper/2508.08424/) (2025)
- [V=arta: A Large-Scale Headline-Generation Dataset for Indic Languages](https://mlsanity.com/paper/2305.05858/) (2023)
- [LexSumm and LexT5: Benchmarking and Modeling Legal Summarization Tasks in English](https://mlsanity.com/paper/2410.09527/) (2024)
- [Pretraining Language Models Using Translationese](https://mlsanity.com/paper/2403.13638/) (2024)
- [Bhaasha, Bhasa, Zaban: A Survey for Low-Resourced Languages in South Asia -- Current Stage and Challenges](https://mlsanity.com/paper/2509.11570/) (2025)
- [NusaWrites: Constructing High-Quality Corpora for Underrepresented and Extremely Low-Resource Languages](https://mlsanity.com/paper/2309.10661/) (2023)
- [Mukhyansh: A Headline Generation Dataset for Indic Languages](https://mlsanity.com/paper/2311.17743/) (2023)
- [Script Gap: Evaluating LLM Triage on Indian Languages in Native vs Romanized Scripts in a Real World Setting](https://mlsanity.com/paper/2512.10780/) (2025)
- [Decoding the Diversity: A Review of the Indic AI Research Landscape](https://mlsanity.com/paper/2406.09559/) (2024)
- [Building pre-train LLM Dataset for the INDIC Languages: a case study on Hindi](https://mlsanity.com/paper/2407.09855/) (2024)
- [A survey of diversity quantification in natural language processing: The why, what, where and how](https://mlsanity.com/paper/2507.20858/) (2025)

## Abstract (arXiv)

Natural Language Generation (NLG) for non-English languages is hampered by the scarcity of datasets in these languages. In this paper, we present the IndicNLG Benchmark, a collection of datasets for benchmarking NLG for 11 Indic languages. We focus on five diverse tasks, namely, biography generation using Wikipedia infoboxes, news headline generation, sentence summarization, paraphrase generation and, question generation. We describe the created datasets and use them to benchmark the performance of several monolingual and multilingual baselines that leverage pre-trained sequence-to-sequence models. Our results exhibit the strong performance of multilingual language-specific pre-trained models, and the utility of models trained on our dataset for other related NLG tasks. Our dataset creation methods can be easily applied to modest-resource languages as they involve simple steps such as scraping news articles and Wikipedia infoboxes, light cleaning, and pivoting through machine translation data. To the best of our knowledge, the IndicNLG Benchmark is the first NLG benchmark for Indic languages and the most diverse multilingual NLG dataset, with approximately 8M examples across 5 tasks and 11 languages. The datasets and models are publicly available at https://ai4bharat.iitm.ac.in/indicnlg-suite.