# Leveraging Transformers for Hate Speech Detection in Conversational Code-Mixed Tweets

arXiv: https://arxiv.org/abs/2112.09986 · HTML: https://mlsanity.com/paper/2112.09986/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 44 references that are in the map.
- First posted on arXiv 18 Dec 2021.

## Builds on (most cited first, 21 of 21)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [HateXplain: A Benchmark Dataset for Explainable Hate Speech Detection](https://mlsanity.com/paper/2012.10289/) (2020)
- [Nuanced Metrics for Measuring Unintended Bias with Real Data for Text Classification](https://mlsanity.com/paper/1903.04561/) (2019)
- [Large Scale Crowdsourcing and Characterization of Twitter Abusive Behavior](https://mlsanity.com/paper/1802.00393/) (2018)
- [Deep Learning for Hate Speech Detection in Tweets](https://mlsanity.com/paper/1706.00188/) (2017)
- [A Benchmark Dataset for Learning to Intervene in Online Hate Speech](https://mlsanity.com/paper/1909.04251/) (2019)
- [Toxicity Detection: Does Context Really Matter?](https://mlsanity.com/paper/2006.00998/) (2020)
- [Toxic Language Detection in Social Media for Brazilian Portuguese: New Dataset and Multilingual Analysis](https://mlsanity.com/paper/2010.04543/) (2020)
- [The Structure of Toxic Conversations on Twitter](https://mlsanity.com/paper/2105.11596/) (2021)
- [Abusive Language Detection with Graph Convolutional Networks](https://mlsanity.com/paper/1904.04073/) (2019)
- [Hate Speech Detection from Code-mixed Hindi-English Tweets Using Deep Learning Models](https://mlsanity.com/paper/1811.05145/) (2018)
- [HABERTOR: An Efficient and Effective Deep Hatespeech Detector](https://mlsanity.com/paper/2010.08865/) (2020)
- [Mind Your Language: Abuse and Offense Detection for Code-Switched Languages](https://mlsanity.com/paper/1809.08652/) (2018)
- [You too Brutus! Trapping Hateful Users in Social Media: Challenges, Solutions & Insights](https://mlsanity.com/paper/2108.00524/) (2021)
- [WLV-RIT at HASOC-Dravidian-CodeMix-FIRE2020: Offensive Language Identification in Code-switched YouTube Comments](https://mlsanity.com/paper/2011.00559/) (2020)
- [Hierarchical CVAE for Fine-Grained Hate Speech Classification](https://mlsanity.com/paper/1809.00088/) (2018)
- [Hostility Detection in Hindi leveraging Pre-Trained Language Models](https://mlsanity.com/paper/2101.05494/) (2021)
- [Trawling for Trolling: A Dataset](https://mlsanity.com/paper/2008.00525/) (2020)
- [Cisco at SemEval-2021 Task 5: What's Toxic?: Leveraging Transformers for Multiple Toxic Span Extraction from Online Comments](https://mlsanity.com/paper/2105.13959/) (2021)
- [IIITG-ADBU@HASOC-Dravidian-CodeMix-FIRE2020: Offensive Content Detection in Code-Mixed Dravidian Text](https://mlsanity.com/paper/2107.14336/) (2021)

## Built on by (most cited first, 3 of 3)

- [CoSyn: Detecting Implicit Hate Speech in Online Conversations Using a Context Synergized Hyperbolic Network](https://mlsanity.com/paper/2303.03387/) (2023)
- [Cordyceps@LT-EDI: Patching Language-Specific Homophobia/Transphobia Classifiers with a Multilingual Understanding](https://mlsanity.com/paper/2309.13561/) (2023)
- [On Importance of Code-Mixed Embeddings for Hate Speech Identification](https://mlsanity.com/paper/2411.18577/) (2024)

## Abstract (arXiv)

In the current era of the internet, where social media platforms are easily accessible for everyone, people often have to deal with threats, identity attacks, hate, and bullying due to their association with a cast, creed, gender, religion, or even acceptance or rejection of a notion. Existing works in hate speech detection primarily focus on individual comment classification as a sequence labeling task and often fail to consider the context of the conversation. The context of a conversation often plays a substantial role when determining the author's intent and sentiment behind the tweet. This paper describes the system proposed by team MIDAS-IIITD for HASOC 2021 subtask 2, one of the first shared tasks focusing on detecting hate speech from Hindi-English code-mixed conversations on Twitter. We approach this problem using neural networks, leveraging the transformer's cross-lingual embeddings and further finetuning them for low-resource hate-speech classification in transliterated Hindi text. Our best performing system, a hard voting ensemble of Indic-BERT, XLM-RoBERTa, and Multilingual BERT, achieved a macro F1 score of 0.7253, placing us first on the overall leaderboard standings.