# How Effectively Can BERT Models Interpret Context and Detect Bengali Communal Violent Text?

arXiv: https://arxiv.org/abs/2506.19831 · HTML: https://mlsanity.com/paper/2506.19831/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 8 of its 29 references that are in the map.
- First posted on arXiv 24 Jun 2025.

## Builds on (most cited first, 8 of 8)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [BanglaBERT: Language Model Pretraining and Benchmarks for Low-Resource Language Understanding Evaluation in Bangla](https://mlsanity.com/paper/2101.00204/) (2021)
- [Hate Speech detection in the Bengali language: A dataset and its baseline evaluation](https://mlsanity.com/paper/2012.09686/) (2020)
- [Classification Benchmarks for Under-resourced Bengali Language based on Multichannel Convolutional-LSTM Network](https://mlsanity.com/paper/2004.07807/) (2020)
- [BanglaParaphrase: A High-Quality Bangla Paraphrase Dataset](https://mlsanity.com/paper/2210.05109/) (2022)
- [Bangla hate speech detection on social media using attention-based recurrent neural network](https://mlsanity.com/paper/2203.16775/) (2022)
- [New Bounds on the Accuracy of Majority Voting for Multi-Class Classification](https://mlsanity.com/paper/2309.09564/) (2023)
- [Mapping Violence: Developing an Extensive Framework to Build a Bangla Sectarian Expression Dataset from Social Media Interactions](https://mlsanity.com/paper/2404.11752/) (2024)

## Abstract (arXiv)

The spread of cyber hatred has led to communal violence, fueling aggression and conflicts between various religious, ethnic, and social groups, posing a significant threat to social harmony. Despite its critical importance, the classification of communal violent text remains an underexplored area in existing research. This study aims to enhance the accuracy of detecting text that incites communal violence, focusing specifically on Bengali textual data sourced from social media platforms. We introduce a fine-tuned BanglaBERT model tailored for this task, achieving a macro F1 score of 0.60. To address the issue of data imbalance, our dataset was expanded by adding 1,794 instances, which facilitated the development and evaluation of a fine-tuned ensemble model. This ensemble model demonstrated an improved performance, achieving a macro F1 score of 0.63, thus highlighting its effectiveness in this domain. In addition to quantitative performance metrics, qualitative analysis revealed instances where the models struggled with context understanding, leading to occasional misclassifications, even when predictions were made with high confidence. Through analyzing the cosine similarity between words, we identified certain limitations in the pre-trained BanglaBERT models, particularly in their ability to distinguish between closely related communal and non-communal terms. To further interpret the model's decisions, we applied LIME, which helped to uncover specific areas where the model struggled in understanding context, contributing to errors in classification. These findings highlight the promise of NLP and interpretability tools in reducing online communal violence. Our work contributes to the growing body of research in communal violence detection and offers a foundation for future studies aiming to refine these techniques for better accuracy and societal impact.