# Bangla Text Classification using Transformers

arXiv: https://arxiv.org/abs/2011.04446 · HTML: https://mlsanity.com/paper/2011.04446/

## Key facts

- Cited by 10 papers in the map (data through 7 Oct 2026).
- First posted on arXiv 9 Nov 2020.

## Built on by (most cited first, 10 of 10)

- [L3Cube-MahaNLP: Marathi Natural Language Processing Datasets, Models, and Library](https://mlsanity.com/paper/2205.14728/) (2022)
- [A Review of Bangla Natural Language Processing Tasks and the Utility of Transformer Models](https://mlsanity.com/paper/2107.03844/) (2021)
- [Parameter-Efficient Fine-Tuning for Low-Resource Languages: A Comparative Study of LLMs for Bengali Hate Speech Detection](https://mlsanity.com/paper/2510.16985/) (2025)
- [MONOVAB : An Annotated Corpus for Bangla Multi-label Emotion Detection](https://mlsanity.com/paper/2309.15670/) (2023)
- [Zero-Shot Multi-Label Classification of Bangla Documents: Large Decoders Vs. Classic Encoders](https://mlsanity.com/paper/2503.02993/) (2025)
- [Bengali Text Classification: An Evaluation of Large Language Model Approaches](https://mlsanity.com/paper/2601.12132/) (2026)
- [Machine and Deep Learning Methods with Manual and Automatic Labelling for News Classification in Bangla Language](https://mlsanity.com/paper/2210.10903/) (2022)
- [A Unified BERT-CNN-BiLSTM Framework for Simultaneous Headline Classification and Sentiment Analysis of Bangla News](https://mlsanity.com/paper/2511.18618/) (2025)
- [Bangla Hate Speech Classification with Fine-tuned Transformer Models](https://mlsanity.com/paper/2512.02845/) (2025)
- [BnBERT-iPET: Sparse Few-Shot Language Modeling for Bengali via Lottery Ticket Pruning](https://mlsanity.com/paper/2608.05104/) (2026)

## Abstract (arXiv)

Text classification has been one of the earliest problems in NLP. Over time the scope of application areas has broadened and the difficulty of dealing with new areas (e.g., noisy social media content) has increased. The problem-solving strategy switched from classical machine learning to deep learning algorithms. One of the recent deep neural network architecture is the Transformer. Models designed with this type of network and its variants recently showed their success in many downstream natural language processing tasks, especially for resource-rich languages, e.g., English. However, these models have not been explored fully for Bangla text classification tasks. In this work, we fine-tune multilingual transformer models for Bangla text classification tasks in different domains, including sentiment analysis, emotion detection, news categorization, and authorship attribution. We obtain the state of the art results on six benchmark datasets, improving upon the previous results by 5-29% accuracy across different tasks.