# Offensive Language Identification in Low-resourced Code-mixed Dravidian languages using Pseudo-labeling

arXiv: https://arxiv.org/abs/2108.12177 · HTML: https://mlsanity.com/paper/2108.12177/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 96 references that are in the map.
- First posted on arXiv 27 Aug 2021.

## Builds on (most cited first, 29 of 29)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [Pointer Sentinel Mixture Models](https://mlsanity.com/paper/1609.07843/) (2016)
- [MuRIL: Multilingual Representations for Indian Languages](https://mlsanity.com/paper/2103.10730/) (2021)
- [SemEval-2020 Task 12: Multilingual Offensive Language Identification in Social Media (OffensEval 2020)](https://mlsanity.com/paper/2006.07235/) (2020)
- [AMMUS : A Survey of Transformer-based Pretrained Models in Natural Language Processing](https://mlsanity.com/paper/2108.05542/) (2021)
- [Using millions of emoji occurrences to learn any-domain representations for detecting sentiment, emotion and sarcasm](https://mlsanity.com/paper/1708.00524/) (2017)
- [Corpus Creation for Sentiment Analysis in Code-Mixed Tamil-English Text](https://mlsanity.com/paper/2006.00206/) (2020)
- [Towards generalisable hate speech detection: a review on obstacles and solutions](https://mlsanity.com/paper/2102.08886/) (2021)
- [A Sentiment Analysis Dataset for Code-Mixed Malayalam-English](https://mlsanity.com/paper/2006.00210/) (2020)
- [Detecting Hate Speech in Social Media](https://mlsanity.com/paper/1712.06427/) (2017)
- [Detecting Offensive Language in Tweets Using Deep Learning](https://mlsanity.com/paper/1801.04433/) (2018)
- [DravidianCodeMix: Sentiment Analysis and Offensive Language Identification Dataset for Dravidian Languages in Code-Mixed Text](https://mlsanity.com/paper/2106.09460/) (2021)
- [Machine Learning Suites for Online Toxicity Detection](https://mlsanity.com/paper/1810.01869/) (2018)
- [SJ_AJ@DravidianLangTech-EACL2021: Task-Adaptive Pre-Training of Multilingual BERT models for Offensive Language Identification](https://mlsanity.com/paper/2102.01051/) (2021)
- [ToxCCIn: Toxic Content Classification with Interpretability](https://mlsanity.com/paper/2103.01328/) (2021)
- [NLP-CUET@DravidianLangTech-EACL2021: Offensive Language Detection from Multilingual Code-Mixed Text using Transformers](https://mlsanity.com/paper/2103.00455/) (2021)
- [Hate-Alert@DravidianLangTech-EACL2021: Ensembling strategies for Transformer-based Offensive language Detection](https://mlsanity.com/paper/2102.10084/) (2021)
- [Hope Speech detection in under-resourced Kannada language](https://mlsanity.com/paper/2108.04616/) (2021)
- [Language Identification in Code-Mixed Data using Multichannel Neural Networks and Context Capture](https://mlsanity.com/paper/1808.07118/) (2018)
- [IIITT@LT-EDI-EACL2021-Hope Speech Detection: There is always Hope in Transformers](https://mlsanity.com/paper/2104.09066/) (2021)
- [UVCE-IIITT@DravidianLangTech-EACL2021: Tamil Troll Meme Classification: You need to Pay more Attention](https://mlsanity.com/paper/2104.09081/) (2021)
- [indicnlp@kgp at DravidianLangTech-EACL2021: Offensive Language Identification in Dravidian Languages](https://mlsanity.com/paper/2102.07150/) (2021)
- [UIT-ISE-NLP at SemEval-2021 Task 5: Toxic Spans Detection with BiLSTM-CRF and ToxicBERT Comment Classification](https://mlsanity.com/paper/2104.10100/) (2021)
- [UniParma at SemEval-2021 Task 5: Toxic Spans Detection Using CharacterBERT and Bag-of-Words Model](https://mlsanity.com/paper/2103.09645/) (2021)

## Built on by (most cited first, 6 of 6)

- [Cross-lingual Offensive Language Detection: A Systematic Review of Datasets, Transfer Approaches and Challenges](https://mlsanity.com/paper/2401.09244/) (2024)
- [Pretrained Transformers for Offensive Language Identification in Tanglish](https://mlsanity.com/paper/2110.02852/) (2021)
- [IIITT@Dravidian-CodeMix-FIRE2021: Transliterate or translate? Sentiment analysis of code-mixed text in Dravidian languages](https://mlsanity.com/paper/2111.07906/) (2021)
- [Findings of the Sentiment Analysis of Dravidian Languages in Code-Mixed Text](https://mlsanity.com/paper/2111.09811/) (2021)
- [Multilingual Text Classification for Dravidian Languages](https://mlsanity.com/paper/2112.01705/) (2021)
- [Hypers at ComMA@ICON: Modelling Aggressiveness, Gender Bias and Communal Bias Identification](https://mlsanity.com/paper/2112.15417/) (2021)

## Abstract (arXiv)

Social media has effectively become the prime hub of communication and digital marketing. As these platforms enable the free manifestation of thoughts and facts in text, images and video, there is an extensive need to screen them to protect individuals and groups from offensive content targeted at them. Our work intends to classify codemixed social media comments/posts in the Dravidian languages of Tamil, Kannada, and Malayalam. We intend to improve offensive language identification by generating pseudo-labels on the dataset. A custom dataset is constructed by transliterating all the code-mixed texts into the respective Dravidian language, either Kannada, Malayalam, or Tamil and then generating pseudo-labels for the transliterated dataset. The two datasets are combined using the generated pseudo-labels to create a custom dataset called CMTRA. As Dravidian languages are under-resourced, our approach increases the amount of training data for the language models. We fine-tune several recent pretrained language models on the newly constructed dataset. We extract the pretrained language embeddings and pass them onto recurrent neural networks. We observe that fine-tuning ULMFiT on the custom dataset yields the best results on the code-mixed test sets of all three languages. Our approach yields the best results among the benchmarked models on Tamil-English, achieving a weighted F1-Score of 0.7934 while scoring competitive weighted F1-Scores of 0.9624 and 0.7306 on the code-mixed test sets of Malayalam-English and Kannada-English, respectively.