mlsanityData to 13 Oct 2026

IndicBART: A Pre-trained Model for Indic Natural Language Generation

arXiv 2109.02903 ↗First posted 7 Sep 2021cs.CL cs.AIcited by 39

Not on the map. Why?

Abstract (arXiv)

In this paper, we study pre-trained sequence-to-sequence models for a group of related languages, with a focus on Indic languages. We present IndicBART, a multilingual, sequence-to-sequence pre-trained model focusing on 11 Indic languages and English. IndicBART utilizes the orthographic similarity between Indic scripts to improve transfer learning between similar Indic languages. We evaluate IndicBART on two NLG tasks: Neural Machine Translation (NMT) and extreme summarization. Our experiments on NMT and extreme summarization show that a model specific to related languages like IndicBART is competitive with large pre-trained models like mBART50 despite being significantly smaller. It also performs well on very low-resource translation scenarios where languages are not included in pre-training or fine-tuning. Script sharing, multilingual training, and better utilization of limited model capacity contribute to the good performance of the compact IndicBART model.

Key facts. Cited by 39 papers in the map (data through 13 Oct 2026). Builds on 33 of its 55 references that are in the map. First posted on arXiv 7 Sep 2021.

Builds on

33 of 55 references.

References · 33

  1. 2017Attention Is All You Needcited by 60,106
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,304
  3. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,884
  4. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,916
  5. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,779
  6. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,979
  7. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,815
  8. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,793
  9. 2016Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translationcited by 1,473
  10. 2018Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarizationcited by 1,312
  11. 2020Multilingual Denoising Pre-training for Neural Machine Translationcited by 1,033
  12. 2016Sequence-Level Knowledge Distillationcited by 770
21 more
  1. 2018Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidatescited by 637
  2. 2021The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translationcited by 556
  3. 2016Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translationcited by 467
  4. 2017A Deep Reinforced Model for Abstractive Summarizationcited by 364
  5. 2021XL-Sum: Large-Scale Multilingual Abstractive Summarization for 44 Languagescited by 346
  6. 2020Multilingual Translation with Extensible Multilingual Pretraining and Finetuningcited by 287
  7. 2019Massively Multilingual Neural Machine Translation in the Wild: Findings and Challengescited by 254
  8. 2019Massively Multilingual Neural Machine Translationcited by 246
  9. 2019Leveraging Pre-trained Checkpoints for Sequence Generation Taskscited by 182
  10. 2021MuRIL: Multilingual Representations for Indian Languagescited by 177
  11. 2021Samanantar: The Largest Publicly Available Parallel Corpora Collection for 11 Indic Languagescited by 133
  12. 2018Data-to-Text Generation with Content Selection and Planningcited by 106
  13. 2020Text-to-Text Pre-Training for Data-to-Text Taskscited by 104
  14. 2019Multilingual Neural Machine Translation with Knowledge Distillationcited by 95
  15. 2019Investigating Multilingual NMT Representations at Scalecited by 68
  16. 2017Transfer Learning across Low-Resource, Related Languages for Neural Machine Translationcited by 58
  17. 2020PMIndia -- A Collection of Parallel Corpora of Languages of Indiacited by 40
  18. 2018Recurrent Stacking of Layers for Compact Neural Machine Translation Modelscited by 38
  19. 2021Exploiting Language Relatedness for Low Web-Resource Language Model Adaptation: An Indic Languages Studycited by 19
  20. 2020A Multilingual Parallel Corpora Collection Effort for Indian Languagescited by 16
  21. 2021YANMTT: Yet Another Neural Machine Translation Toolkitcited by 7

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2022: 7; 2023: 13; 2024: 8; 2025: 8; 2026: 3.

39 papers.

Sort:

By area:

  1. 2022Pretrained Language Models for Text Generation: A Surveycited by 122
  2. 2022Towards Leaving No Indic Language Behind: Building Monolingual Corpora, Benchmark and Models for Indic Languagescited by 99
  3. 2022IndicNLG Benchmark: Multilingual Datasets for Diverse NLG Tasks in Indic Languagescited by 42
  4. 2024SEACrowd: A Multilingual Multimodal Data Hub and Benchmark Suite for Southeast Asian Languagescited by 41
  5. 2022BanglaNLG and BanglaT5: Benchmarks and Resources for Evaluating Low-Resource Natural Language Generation in Banglacited by 40
  6. 2024RomanSetu: Efficiently unlocking multilingual capabilities of Large Language Models via Romanizationcited by 29
  7. 2022IndicMT Eval: A Dataset to Meta-Evaluate Machine Translation metrics for Indian Languagescited by 27
  8. 2022IndicXNLI: Evaluating Multilingual Inference for Indian Languagescited by 26
  9. 2023PMIndiaSum: Multilingual and Cross-lingual Headline Summarization for Languages in Indiacited by 11
  10. 2025Llama-3-Nanda-10B-Chat: An Open Generative Large Language Model for Hindicited by 10
  11. 2023V=arta: A Large-Scale Headline-Generation Dataset for Indic Languagescited by 8
  12. 2025The State of Large Language Models for African Languages: Progress and Challengescited by 8
27 more
  1. 2024Table Question Answering for Low-resourced Indic Languagescited by 7
  2. 2024Too Late to Train, Too Early To Use? A Study on Necessity and Viability of Low-Resource Bengali LLMscited by 6
  3. 2025Chitranuvad: Adapting Multi-Lingual LLMs for Multimodal Translationcited by 6
  4. 2023An Empirical Study of Leveraging Knowledge Distillation for Compressing Multilingual Neural Machine Translation Modelscited by 5
  5. 2023Mukhyansh: A Headline Generation Dataset for Indic Languagescited by 5
  6. 2023IndicIRSuite: Multilingual Dataset and Neural Information Models for Indian Languagescited by 5
  7. 2023Decomposed Prompting for Machine Translation Between Related Languages using Large Language Modelscited by 3
  8. 2023S=amayik: A Benchmark and Dataset for English-Sanskrit Translationcited by 3
  9. 2024RoundTripOCR: A Data Generation Technique for Enhancing Post-OCR Error Correction in Low-Resource Devanagari Languagescited by 3
  10. 2023Summarizing Indian Languages using Multilingual Transformers based Modelscited by 2
  11. 2023Sequence-to-Sequence Spanish Pre-trained Language Modelscited by 2
  12. 2023The Less the Merrier? Investigating Language Representation in Multilingual Modelscited by 2
  13. 2024AgriLLM: Harnessing Transformers for Farmer Queriescited by 2
  14. 2025BanglaByT5: Byte-Level Modelling for Banglacited by 2
  15. 2022Implementing Deep Learning-Based Approaches for Article Summarization in Indian Languagescited by 1
  16. 2023Evaluating Inter-Bilingual Semantic Parsing for Indian Languagescited by 1
  17. 2023Multilingual Bias Detection and Mitigation for Indian Languagescited by 1
  18. 2024Machine Translation with Large Language Models: Decoder Only vs. Encoder-Decodercited by 1
  19. 2024L3Cube-MahaSum: A Comprehensive Dataset and BART Models for Abstractive Text Summarization in Marathicited by 1
  20. 2025Regional Tiny Stories: Using Small Models to Compare Language Learning and Tokenizer Performancecited by 1
  21. 2023Machine Translation by Projecting Text into the Same Phonetic-Orthographic Space Using a Common Encodingcited by 0
  22. 2025Revisiting Metric Reliability for Fine-grained Evaluation of Machine Translation and Summarization in Indian Languagescited by 0
  23. 2025SentiMaithili: A Benchmark Dataset for Sentiment and Reason Generation for the Low-Resource Maithili Languagecited by 0
  24. 2025Yes-MT's Submission to the Low-Resource Indic Language Translation Shared Task in WMT 2024cited by 0
  25. 2026Progressive Training for Explainable Citation-Grounded Dialogue: Reducing Hallucination to Zero in English-Hindi LLMscited by 0
  26. 2026Enhancing ASR Performance in the Medical Domain for Dravidian Languagescited by 0
  27. 2026BhashaSetu: A Data-Centric Approach to Low-Resource Machine Translationcited by 0