# A Twitter BERT Approach for Offensive Language Detection in Marathi

arXiv: https://arxiv.org/abs/2212.10039 · HTML: https://mlsanity.com/paper/2212.10039/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 23 references that are in the map.
- First posted on arXiv 20 Dec 2022.

## Builds on (most cited first, 14 of 14)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [HateBERT: Retraining BERT for Abusive Language Detection in English](https://mlsanity.com/paper/2010.12472/) (2020)
- [MuRIL: Multilingual Representations for Indian Languages](https://mlsanity.com/paper/2103.10730/) (2021)
- [L3Cube-MahaCorpus and MahaBERT: Marathi Monolingual Corpus, Marathi BERT Language Models, and Resources](https://mlsanity.com/paper/2202.01159/) (2022)
- [FBERT: A Neural Transformer for Identifying Offensive Content](https://mlsanity.com/paper/2109.05074/) (2021)
- [Cross-lingual Offensive Language Identification for Low Resource Languages: The Case of Marathi](https://mlsanity.com/paper/2109.03552/) (2021)
- [L3Cube-MahaHate: A Tweet-based Marathi Hate Speech Detection Dataset and BERT models](https://mlsanity.com/paper/2203.13778/) (2022)
- [Overview of the HASOC Subtrack at FIRE 2021: Hate Speech and Offensive Content Identification in English and Indo-Aryan Languages](https://mlsanity.com/paper/2112.09301/) (2021)
- [Mono vs Multilingual BERT for Hate Speech Detection and Text Classification: A Case Study in Marathi](https://mlsanity.com/paper/2204.08669/) (2022)
- [Overview of the HASOC Subtrack at FIRE 2022: Offensive Language Identification in Marathi](https://mlsanity.com/paper/2211.10163/) (2022)
- [Predicting the Type and Target of Offensive Social Media Posts in Marathi](https://mlsanity.com/paper/2211.12570/) (2022)
- [Spread Love Not Hate: Undermining the Importance of Hateful Pre-training for Hate Speech Detection](https://mlsanity.com/paper/2210.04267/) (2022)
- [A Review of Challenges in Machine Learning based Automated Hate Speech Detection](https://mlsanity.com/paper/2209.05294/) (2022)

## Built on by (most cited first, 3 of 3)

- [A Survey of Large Language Models for Healthcare: from Data, Technology, and Applications to Accountability and Ethics](https://mlsanity.com/paper/2310.05694/) (2023)
- [A Survey on Automatic Online Hate Speech Detection in Low-Resource Languages](https://mlsanity.com/paper/2411.19017/) (2024)
- [Harnessing Pre-Trained Sentence Transformers for Offensive Language Detection in Indian Languages](https://mlsanity.com/paper/2310.02249/) (2023)

## Abstract (arXiv)

Automated offensive language detection is essential in combating the spread of hate speech, particularly in social media. This paper describes our work on Offensive Language Identification in low resource Indic language Marathi. The problem is formulated as a text classification task to identify a tweet as offensive or non-offensive. We evaluate different mono-lingual and multi-lingual BERT models on this classification task, focusing on BERT models pre-trained with social media datasets. We compare the performance of MuRIL, MahaTweetBERT, MahaTweetBERT-Hateful, and MahaBERT on the HASOC 2022 test set. We also explore external data augmentation from other existing Marathi hate speech corpus HASOC 2021 and L3Cube-MahaHate. The MahaTweetBERT, a BERT model, pre-trained on Marathi tweets when fine-tuned on the combined dataset (HASOC 2021 + HASOC 2022 + MahaHate), outperforms all models with an F1 score of 98.43 on the HASOC 2022 test set. With this, we also provide a new state-of-the-art result on HASOC 2022 / MOLD v2 test set.