# NLPineers@ NLU of Devanagari Script Languages 2025: Hate Speech Detection using Ensembling of BERT-based models

arXiv: https://arxiv.org/abs/2412.08163 · HTML: https://mlsanity.com/paper/2412.08163/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 24 references that are in the map.
- First posted on arXiv 11 Dec 2024.

## Builds on (most cited first, 11 of 11)

- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [Multilingual Translation with Extensible Multilingual Pretraining and Finetuning](https://mlsanity.com/paper/2008.00401/) (2020)
- [Learning Word Vectors for 157 Languages](https://mlsanity.com/paper/1802.06893/) (2018)
- [HateBERT: Retraining BERT for Abusive Language Detection in English](https://mlsanity.com/paper/2010.12472/) (2020)
- [MuRIL: Multilingual Representations for Indian Languages](https://mlsanity.com/paper/2103.10730/) (2021)
- [L3CubeMahaSent: A Marathi Tweet-based Sentiment Analysis Dataset](https://mlsanity.com/paper/2103.11408/) (2021)
- [Data Bootstrapping Approaches to Improve Low Resource Abusive Language Detection for Indic Languages](https://mlsanity.com/paper/2204.12543/) (2022)
- [Itihasa: A large-scale corpus for Sanskrit to English translation](https://mlsanity.com/paper/2106.03269/) (2021)
- [Uncovering Political Hate Speech During Indian Election Campaign: A New Low-Resource Dataset and Baselines](https://mlsanity.com/paper/2306.14764/) (2023)
- [Can Perplexity Predict Fine-tuning Performance? An Investigation of Tokenization Effects on Sequential Language Models for Nepali](https://mlsanity.com/paper/2404.18071/) (2024)
- [Linguistic Taboos and Euphemisms in Nepali](https://mlsanity.com/paper/2007.13798/) (2020)

## Built on by (most cited first, 3 of 3)

- [Code-Mixed Telugu-English Hate Speech Detection](https://mlsanity.com/paper/2502.10632/) (2025)
- [Cascading versus Joint Modeling for Hierarchical Offensive Language Detection](https://mlsanity.com/paper/2607.16790/) (2026)
- [ShriNep@EEUCA 2026: RAKSHAK - Multi-Task DeBERTa with Rationale Distillation and Jigsaw-Augmented Training for Toxic Intent Classification](https://mlsanity.com/paper/2607.20450/) (2026)

## Abstract (arXiv)

This paper explores hate speech detection in Devanagari-scripted languages, focusing on Hindi and Nepali, for Subtask B of the CHIPSAL@COLING 2025 Shared Task. Using a range of transformer-based models such as XLM-RoBERTa, MURIL, and IndicBERT, we examine their effectiveness in navigating the nuanced boundary between hate speech and free expression. Our best performing model, implemented as ensemble of multilingual BERT models achieve Recall of 0.7762 (Rank 3/31 in terms of recall) and F1 score of 0.6914 (Rank 17/31). To address class imbalance, we used backtranslation for data augmentation, and cosine similarity to preserve label consistency after augmentation. This work emphasizes the need for hate speech detection in Devanagari-scripted languages and presents a foundation for further research.