# Hate-Alert@DravidianLangTech-EACL2021: Ensembling strategies for Transformer-based Offensive language Detection

arXiv: https://arxiv.org/abs/2102.10084 · HTML: https://mlsanity.com/paper/2102.10084/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 41 references that are in the map.
- First posted on arXiv 19 Feb 2021.

## Builds on (most cited first, 13 of 13)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [Enriching Word Vectors with Subword Information](https://mlsanity.com/paper/1607.04606/) (2016)
- [Automated Hate Speech Detection and the Problem of Offensive Language](https://mlsanity.com/paper/1703.04009/) (2017)
- [Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyond](https://mlsanity.com/paper/1812.10464/) (2018)
- [BERTs of a feather do not generalize together: Large variability in generalization across models with similar test set performance](https://mlsanity.com/paper/1911.02969/) (2019)
- [Corpus Creation for Sentiment Analysis in Code-Mixed Tamil-English Text](https://mlsanity.com/paper/2006.00206/) (2020)
- [Offensive Language Identification in Greek](https://mlsanity.com/paper/2003.07459/) (2020)
- [A Sentiment Analysis Dataset for Code-Mixed Malayalam-English](https://mlsanity.com/paper/2006.00210/) (2020)
- [Offensive Language and Hate Speech Detection for Danish](https://mlsanity.com/paper/1908.04531/) (2019)
- [Processing South Asian Languages Written in the Latin Script: the Dakshina Dataset](https://mlsanity.com/paper/2007.01176/) (2020)

## Built on by (most cited first, 8 of 8)

- [Vietnamese Hate and Offensive Detection using PhoBERT-CNN and Social Media Streaming Data](https://mlsanity.com/paper/2206.00524/) (2022)
- [Cyberbullying Detection for Low-resource Languages and Dialects: Review of the State of the Art](https://mlsanity.com/paper/2308.15745/) (2023)
- [EkoHate: Abusive Language and Hate Speech Detection for Code-switched Political Discussions on Nigerian Twitter](https://mlsanity.com/paper/2404.18180/) (2024)
- [Offensive Language Identification in Low-resourced Code-mixed Dravidian languages using Pseudo-labeling](https://mlsanity.com/paper/2108.12177/) (2021)
- [Towards Offensive Language Identification for Tamil Code-Mixed YouTube Comments and Posts](https://mlsanity.com/paper/2108.10939/) (2021)
- [bitsa_nlp@LT-EDI-ACL2022: Leveraging Pretrained Language Models for Detecting Homophobia and Transphobia in Social Media Comments](https://mlsanity.com/paper/2203.14267/) (2022)
- [Developing Successful Shared Tasks on Offensive Language Identification for Dravidian Languages](https://mlsanity.com/paper/2111.03375/) (2021)
- [Multilingual Text Classification for Dravidian Languages](https://mlsanity.com/paper/2112.01705/) (2021)

## Abstract (arXiv)

Social media often acts as breeding grounds for different forms of offensive content. For low resource languages like Tamil, the situation is more complex due to the poor performance of multilingual or language-specific models and lack of proper benchmark datasets. Based on this shared task, Offensive Language Identification in Dravidian Languages at EACL 2021, we present an exhaustive exploration of different transformer models, We also provide a genetic algorithm technique for ensembling different models. Our ensembled models trained separately for each language secured the first position in Tamil, the second position in Kannada, and the first position in Malayalam sub-tasks. The models and codes are provided.