# Contextual Hate Speech Detection in Code Mixed Text using Transformer Based Approaches

arXiv: https://arxiv.org/abs/2110.09338 · HTML: https://mlsanity.com/paper/2110.09338/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 23 references that are in the map.
- First posted on arXiv 18 Oct 2021.

## Builds on (most cited first, 11 of 11)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [How multilingual is Multilingual BERT?](https://mlsanity.com/paper/1906.01502/) (2019)
- [Detecting Online Hate Speech Using Context Aware Models](https://mlsanity.com/paper/1710.07395/) (2017)
- [Hate Speech Detection: A Solved Problem? The Challenging Case of Long Tail on Twitter](https://mlsanity.com/paper/1803.03662/) (2018)
- [Deep Learning for Hindi Text Classification: A Comparison](https://mlsanity.com/paper/2001.10340/) (2020)
- [Abusive Language Detection with Graph Convolutional Networks](https://mlsanity.com/paper/1904.04073/) (2019)
- [Evaluating Deep Learning Approaches for Covid19 Fake News Detection](https://mlsanity.com/paper/2101.04012/) (2021)
- [Transfer Learning for Hate Speech Detection in Social Media](https://mlsanity.com/paper/1906.03829/) (2019)
- [Evaluation of Deep Learning Models for Hostility Detection in Hindi Text](https://mlsanity.com/paper/2101.04144/) (2021)
- [Evaluating Input Representation for Language Identification in Hindi-English Code Mixed Text](https://mlsanity.com/paper/2011.11263/) (2020)

## Built on by (most cited first, 4 of 4)

- [L3Cube-MahaCorpus and MahaBERT: Marathi Monolingual Corpus, Marathi BERT Language Models, and Resources](https://mlsanity.com/paper/2202.01159/) (2022)
- [L3Cube-HingCorpus and HingBERT: A Code Mixed Hindi-English Dataset and BERT Language Models](https://mlsanity.com/paper/2204.08398/) (2022)
- [Leveraging Language Identification to Enhance Code-Mixed Text Classification](https://mlsanity.com/paper/2306.04964/) (2023)
- [On Importance of Code-Mixed Embeddings for Hate Speech Identification](https://mlsanity.com/paper/2411.18577/) (2024)

## Abstract (arXiv)

In the recent past, social media platforms have helped people in connecting and communicating to a wider audience. But this has also led to a drastic increase in cyberbullying. It is essential to detect and curb hate speech to keep the sanity of social media platforms. Also, code mixed text containing more than one language is frequently used on these platforms. We, therefore, propose automated techniques for hate speech detection in code mixed text from scraped Twitter. We specifically focus on code mixed English-Hindi text and transformer-based approaches. While regular approaches analyze the text independently, we also make use of content text in the form of parent tweets. We try to evaluate the performances of multilingual BERT and Indic-BERT in single-encoder and dual-encoder settings. The first approach is to concatenate the target text and context text using a separator token and get a single representation from the BERT model. The second approach encodes the two texts independently using a dual BERT encoder and the corresponding representations are averaged. We show that the dual-encoder approach using independent representations yields better performance. We also employ simple ensemble methods to further improve the performance. Using these methods we report the best F1 score of 73.07% on the HASOC 2021 ICHCL code mixed data set.