mlsanityData to 14 Oct 2026

BanglaBERT: Language Model Pretraining and Benchmarks for Low-Resource Language Understanding Evaluation in Bangla

arXiv 2101.00204 ↗First posted 1 Jan 2021cs.CLcited by 125

Not on the map. Why?

Abstract (arXiv)

In this work, we introduce BanglaBERT, a BERT-based Natural Language Understanding (NLU) model pretrained in Bangla, a widely spoken yet low-resource language in the NLP literature. To pretrain BanglaBERT, we collect 27.5 GB of Bangla pretraining data (dubbed `Bangla2B+') by crawling 110 popular Bangla sites. We introduce two downstream task datasets on natural language inference and question answering and benchmark on four diverse NLU tasks covering text classification, sequence labeling, and span prediction. In the process, we bring them under the first-ever Bangla Language Understanding Benchmark (BLUB). BanglaBERT achieves state-of-the-art results outperforming multilingual and monolingual models. We are making the models, datasets, and a leaderboard publicly available at https://github.com/csebuetnlp/banglabert to advance Bangla NLP.

Key facts. Cited by 125 papers in the map (data through 14 Oct 2026). Builds on 26 of its 58 references that are in the map. First posted on arXiv 1 Jan 2021.

Builds on

26 of 58 references.

References · 26

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  4. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,899
  5. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
  6. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,811
  7. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,790
  8. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,682
  9. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,554
  10. 2016Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translationcited by 1,475
  11. 2019Cross-lingual Language Model Pretrainingcited by 1,090
  12. 2015Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Bookscited by 1,043
14 more
  1. 2018XNLI: Evaluating Cross-lingual Sentence Representationscited by 915
  2. 2016Bag of Tricks for Efficient Text Classificationcited by 876
  3. 2019SpanBERT: Improving Pre-training by Representing and Predicting Spanscited by 834
  4. 2020The State and Fate of Linguistic Diversity and Inclusion in the NLP Worldcited by 798
  5. 2020Language-agnostic BERT Sentence Embeddingcited by 627
  6. 2019CCNet: Extracting High Quality Monolingual Datasets from Web Crawl Datacited by 548
  7. 2020TyDi QA: A Benchmark for Information-Seeking Question Answering in Typologically Diverse Languagescited by 488
  8. 2018Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyondcited by 434
  9. 2019CamemBERT: a Tasty French Language Modelcited by 368
  10. 2020AraBERT: Transformer-based Model for Arabic Language Understandingcited by 362
  11. 2020Are All Languages Created Equal in Multilingual BERT?cited by 207
  12. 2019Cross-Lingual Ability of Multilingual BERT: An Empirical Studycited by 182
  13. 2019FlauBERT: Unsupervised Language Model Pre-training for Frenchcited by 153
  14. 2020Not Low-Resource Anymore: Aligner Ensembling, Batch Filtering, and New Datasets for Bengali-English Machine Translationcited by 53

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 4; 2023: 28; 2024: 28; 2025: 34; 2026: 31.

125 papers.

Sort:

By area:

  1. 2022BanglaNLG and BanglaT5: Benchmarks and Resources for Evaluating Low-Resource Natural Language Generation in Banglacited by 42
  2. 2022L3Cube-HindBERT and DevBERT: Pre-Trained BERT Transformer models for Devanagari based Hindi and Marathi Languagescited by 35
  3. 2023BenLLMEval: A Comprehensive Evaluation into the Potentials and Pitfalls of Large Language Models on Bengali NLPcited by 23
  4. 2023Zero- and Few-Shot Prompting with LLMs: A Comparative Study with Fine-tuned Models for Bangla Sentiment Analysiscited by 20
  5. 2023Sabi'a: Portuguese Large Language Modelscited by 19
  6. 2023BanglaAbuseMeme: A Dataset for Bengali Abusive Meme Classificationcited by 18
  7. 2025TigerLLM - A Family of Bangla Large Language Modelscited by 16
  8. 2022BanglaParaphrase: A High-Quality Bangla Paraphrase Datasetcited by 15
  9. 2023BanglaBook: A Large-scale Bangla Dataset for Sentiment Analysis from Book Reviewscited by 14
  10. 2024Social Bias in Large Language Models For Bangla: An Empirical Study on Gender and Religious Biascited by 13
  11. 2023Vacaspati: A Diverse Corpus of Bangla Literaturecited by 10
  12. 2024BnSentMix: A Diverse Bengali-English Code-Mixed Dataset for Sentiment Analysiscited by 10
38 more
  1. 2024BanTH: A Multi-label Hate Speech Detection Dataset for Transliterated Banglacited by 10
  2. 2025TituLLMs: A Family of Bangla LLMs with Comprehensive Benchmarkingcited by 9
  3. 2025Script Fragmentation and Format: What Drives the English-Bengali Performance Gap in Open LLMs?cited by 9
  4. 2023SentiGOLD: A Large Bangla Gold Standard Multi-Domain Sentiment Analysis Dataset and its Evaluationcited by 8
  5. 2023Cyberbullying Detection for Low-resource Languages and Dialects: Review of the State of the Artcited by 8
  6. 2023BLP-2023 Task 2: Sentiment Analysiscited by 7
  7. 2023SentMix-3L: A Bangla-English-Hindi Code-Mixed Dataset for Sentiment Analysiscited by 7
  8. 2023Crosslingual Retrieval Augmented In-context Learning for Banglacited by 7
  9. 2024Too Late to Train, Too Early To Use? A Study on Necessity and Viability of Low-Resource Bengali LLMscited by 7
  10. 2024Table Question Answering for Low-resourced Indic Languagescited by 7
  11. 2024Harnessing Large Language Models Over Transformer Models for Detecting Bengali Depressive Social Media Text: A Comprehensive Studycited by 6
  12. 2025Performance Evaluation of Large Language Models in Bangla Consumer Health Query Summarizationcited by 6
  13. 2025TigerCoder: A Novel Suite of LLMs for Code Generation in Banglacited by 6
  14. 2025LLM-Based Multi-Task Bangla Hate Speech Detection: Type, Severity, and Targetcited by 6
  15. 2025Read Between the Lines: A Benchmark for Uncovering Political Bias in Bangla News Articlescited by 6
  16. 2023OffMix-3L: A Novel Code-Mixed Dataset in Bangla-English-Hindi for Offensive Language Identificationcited by 5
  17. 2024BanglaAutoKG: Automatic Bangla Knowledge Graph Construction with Semantic Neural Graph Filteringcited by 5
  18. 2024Multiple Sources are Better Than One: Incorporating External Knowledge in Low-Resource Glossingcited by 5
  19. 2024An Empirical Study of Gendered Stereotypes in Emotional Attributes for Bangla in Multilingual Large Language Modelscited by 5
  20. 2024Disentangling Singlish Discourse Particles with Task-Driven Representationcited by 5
  21. 2024ChitroJera: A Regionally Relevant Visual Question Answering Dataset for Banglacited by 5
  22. 2024Multi-ToM: Evaluating Multilingual Theory of Mind Capabilities in Large Language Modelscited by 5
  23. 2026Cross-Lingual Sentiment Misalignment: Auditing Multilingual Language Models for Inversion Risk, Dialectal Representation, and Affective Stabilitycited by 5
  24. 2023Mixed-Distil-BERT: Code-mixed Language Modeling for Bangla, English, and Hindicited by 4
  25. 2024An Empirical Study on the Characteristics of Bias upon Context Length Variation for Banglacited by 4
  26. 2024BanglaQuAD: A Bengali Open-domain Question Answering Datasetcited by 4
  27. 2024Monolingual and Multilingual Misinformation Detection for Low-Resource Languages: A Comprehensive Surveycited by 4
  28. 2025ANCHOLIK-NER: A Benchmark Dataset for Bangla Regional Named Entity Recognitioncited by 4
  29. 2023Bengali Fake Review Detection using Semi-supervised Generative Adversarial Networkscited by 3
  30. 2023Intent Detection and Slot Filling for Home Assistants: Dataset and Analysis for Bangla and Sylheticited by 3
  31. 2023LowResource at BLP-2023 Task 2: Leveraging BanglaBert for Low Resource Sentiment Analysis of Bangla Languagecited by 3
  32. 2023Bengali Intent Classification with Generative Adversarial BERTcited by 3
  33. 2024A Comparative Analysis of Noise Reduction Methods in Sentiment Analysis on Noisy Bangla Textscited by 3
  34. 2025Empowering Bengali Education with AI: Solving Bengali Math Word Problems through Transformer Modelscited by 3
  35. 2025From Scarcity to Capability: Empowering Fake News Detection in Low-Resource Languages with LLMscited by 3
  36. 2025ANUBHUTI: A Comprehensive Corpus For Sentiment Analysis In Bangla Regional Languagescited by 3
  37. 2025Parameter-Efficient Fine-Tuning for Low-Resource Languages: A Comparative Study of LLMs for Bengali Hate Speech Detectioncited by 3
  38. 2023On Evaluation of Bangla Word Analogiescited by 2

50 most cited of 125 All 125 (JSON)