# BanglaNLG and BanglaT5: Benchmarks and Resources for Evaluating Low-Resource Natural Language Generation in Bangla

arXiv: https://arxiv.org/abs/2205.11081 · HTML: https://mlsanity.com/paper/2205.11081/

## Key facts

- Cited by 42 papers in the map (data through 14 Oct 2026).
- Builds on 49 of its 67 references that are in the map.
- First posted on arXiv 23 May 2022.

## Builds on (most cited first, 30 of 49)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing](https://mlsanity.com/paper/1808.06226/) (2018)
- [mT5: A massively multilingual pre-trained text-to-text transformer](https://mlsanity.com/paper/2010.11934/) (2020)
- [GLU Variants Improve Transformer](https://mlsanity.com/paper/2002.05202/) (2020)
- [A Call for Clarity in Reporting BLEU Scores](https://mlsanity.com/paper/1804.08771/) (2018)
- [Know What You Don't Know: Unanswerable Questions for SQuAD](https://mlsanity.com/paper/1806.03822/) (2018)
- [Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation](https://mlsanity.com/paper/1609.08144/) (2016)
- [Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarization](https://mlsanity.com/paper/1808.08745/) (2018)
- [Multilingual Denoising Pre-training for Neural Machine Translation](https://mlsanity.com/paper/2001.08210/) (2020)
- [On Faithfulness and Factuality in Abstractive Summarization](https://mlsanity.com/paper/2005.00661/) (2020)
- [DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generation](https://mlsanity.com/paper/1911.00536/) (2019)
- [Personalizing Dialogue Agents: I have a dog, do you have pets too?](https://mlsanity.com/paper/1801.07243/) (2018)
- [The State and Fate of Linguistic Diversity and Inclusion in the NLP World](https://mlsanity.com/paper/2004.09095/) (2020)
- [XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalization](https://mlsanity.com/paper/2003.11080/) (2020)
- [CoQA: A Conversational Question Answering Challenge](https://mlsanity.com/paper/1808.07042/) (2018)
- [DailyDialog: A Manually Labelled Multi-turn Dialogue Dataset](https://mlsanity.com/paper/1710.03957/) (2017)
- [The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translation](https://mlsanity.com/paper/2106.03193/) (2021)
- [CCNet: Extracting High Quality Monolingual Datasets from Web Crawl Data](https://mlsanity.com/paper/1911.00359/) (2019)
- [TyDi QA: A Benchmark for Information-Seeking Question Answering in Typologically Diverse Languages](https://mlsanity.com/paper/2003.05002/) (2020)

## Built on by (most cited first, 30 of 42)

- [BenLLMEval: A Comprehensive Evaluation into the Potentials and Pitfalls of Large Language Models on Bengali NLP](https://mlsanity.com/paper/2309.13173/) (2023)
- [Dolphin: A Challenging and Diverse Benchmark for Arabic NLG](https://mlsanity.com/paper/2305.14989/) (2023)
- [Vashantor: A Large-scale Multilingual Benchmark Dataset for Automated Translation of Bangla Regional Dialects to Bangla Language](https://mlsanity.com/paper/2311.11142/) (2023)
- [Where Are We? Evaluating LLM Performance on African Languages](https://mlsanity.com/paper/2502.19582/) (2025)
- [TituLLMs: A Family of Bangla LLMs with Comprehensive Benchmarking](https://mlsanity.com/paper/2502.11187/) (2025)
- [Too Late to Train, Too Early To Use? A Study on Necessity and Viability of Low-Resource Bengali LLMs](https://mlsanity.com/paper/2407.00416/) (2024)
- [Performance Evaluation of Large Language Models in Bangla Consumer Health Query Summarization](https://mlsanity.com/paper/2505.05070/) (2025)
- [Paramanu: Compact and Competitive Monolingual Language Models for Low-Resource Morphologically Rich Indian Languages](https://mlsanity.com/paper/2401.18034/) (2024)
- [Low-Resource Counterspeech Generation for Indic Languages: The Case of Bengali and Hindi](https://mlsanity.com/paper/2402.07262/) (2024)
- [BanglaAutoKG: Automatic Bangla Knowledge Graph Construction with Semantic Neural Graph Filtering](https://mlsanity.com/paper/2404.03528/) (2024)
- [An Empirical Study of Gendered Stereotypes in Emotional Attributes for Bangla in Multilingual Large Language Models](https://mlsanity.com/paper/2407.06432/) (2024)
- [Decoding the Diversity: A Review of the Indic AI Research Landscape](https://mlsanity.com/paper/2406.09559/) (2024)
- [An Empirical Study on the Characteristics of Bias upon Context Length Variation for Bangla](https://mlsanity.com/paper/2406.17375/) (2024)
- [Intent Detection and Slot Filling for Home Assistants: Dataset and Analysis for Bangla and Sylheti](https://mlsanity.com/paper/2310.10935/) (2023)
- [LowResource at BLP-2023 Task 2: Leveraging BanglaBert for Low Resource Sentiment Analysis of Bangla Language](https://mlsanity.com/paper/2311.12735/) (2023)
- [Empowering Bengali Education with AI: Solving Bengali Math Word Problems through Transformer Models](https://mlsanity.com/paper/2501.02599/) (2025)
- [Bridging Dialects: Translating Standard Bangla to Regional Variants Using Neural Models](https://mlsanity.com/paper/2501.05749/) (2025)
- [Transcribing Bengali Text with Regional Dialects to IPA using District Guided Tokens](https://mlsanity.com/paper/2403.17407/) (2024)
- [BanglaDialecto: An End-to-End AI-Powered Regional Speech Standardization](https://mlsanity.com/paper/2411.10879/) (2024)
- [Zero-Shot Multi-Label Classification of Bangla Documents: Large Decoders Vs. Classic Encoders](https://mlsanity.com/paper/2503.02993/) (2025)
- [BanglaByT5: Byte-Level Modelling for Bangla](https://mlsanity.com/paper/2505.17102/) (2025)
- [CRaFT: An Explanation-Based Framework for Evaluating Cultural Reasoning in Multilingual Language Models](https://mlsanity.com/paper/2510.14014/) (2025)
- [Reliability Gated Multi-Teacher Distillation for Low Resource Abstractive Summarization](https://mlsanity.com/paper/2604.03192/) (2026)
- [Milestones in Bengali Sentiment Analysis leveraging Transformer-models: Fundamentals, Challenges and Future Directions](https://mlsanity.com/paper/2401.07847/) (2024)
- [BeliN: A Novel Corpus for Bengali Religious News Headline Generation using Contextual Feature Fusion](https://mlsanity.com/paper/2501.01069/) (2025)
- [Survey of NLU Benchmarks Diagnosing Linguistic Phenomena: Why not Standardize Diagnostics Benchmarks?](https://mlsanity.com/paper/2507.20419/) (2025)
- [BanglaNirTox: A Large-scale Parallel Corpus for Explainable AI in Bengali Text Detoxification](https://mlsanity.com/paper/2511.01512/) (2025)
- [BengaliFig: A Low-Resource Challenge for Figurative and Culturally Grounded Reasoning in Bengali](https://mlsanity.com/paper/2511.20399/) (2025)
- [Modeling Romanized Hindi and Bengali: Dataset Creation and Multilingual LLM Integration](https://mlsanity.com/paper/2511.22769/) (2025)
- [KrishokChat: A Provenance-Traceable Multi-Task Bengali Agricultural Benchmark with Safety-Critical Chemical Advisory](https://mlsanity.com/paper/2606.29243/) (2026)

## Abstract (arXiv)

This work presents BanglaNLG, a comprehensive benchmark for evaluating natural language generation (NLG) models in Bangla, a widely spoken yet low-resource language. We aggregate six challenging conditional text generation tasks under the BanglaNLG benchmark, introducing a new dataset on dialogue generation in the process. Furthermore, using a clean corpus of 27.5 GB of Bangla data, we pretrain BanglaT5, a sequence-to-sequence Transformer language model for Bangla. BanglaT5 achieves state-of-the-art performance in all of these tasks, outperforming several multilingual models by up to 9% absolute gain and 32% relative gain. We are making the new dialogue dataset and the BanglaT5 model publicly available at https://github.com/csebuetnlp/BanglaNLG in the hope of advancing future research on Bangla NLG.