# DravidianCodeMix: Sentiment Analysis and Offensive Language Identification Dataset for Dravidian Languages in Code-Mixed Text

arXiv: https://arxiv.org/abs/2106.09460 · HTML: https://mlsanity.com/paper/2106.09460/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 91 references that are in the map.
- First posted on arXiv 17 Jun 2021.

## Builds on (most cited first, 13 of 13)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [Cross-lingual Language Model Pretraining](https://mlsanity.com/paper/1901.07291/) (2019)
- [Predicting the Type and Target of Offensive Posts in Social Media](https://mlsanity.com/paper/1902.09666/) (2019)
- [SemEval-2020 Task 12: Multilingual Offensive Language Identification in Social Media (OffensEval 2020)](https://mlsanity.com/paper/2006.07235/) (2020)
- [CharacterBERT: Reconciling ELMo and BERT for Word-Level Open-Vocabulary Representations From Characters](https://mlsanity.com/paper/2010.10392/) (2020)
- [Corpus Creation for Sentiment Analysis in Code-Mixed Tamil-English Text](https://mlsanity.com/paper/2006.00206/) (2020)
- [SemEval-2020 Task 9: Overview of Sentiment Analysis of Code-Mixed Tweets](https://mlsanity.com/paper/2008.04277/) (2020)
- [A Sentiment Analysis Dataset for Code-Mixed Malayalam-English](https://mlsanity.com/paper/2006.00210/) (2020)
- [Duluth UROP at SemEval-2018 Task 2: Multilingual Emoji Prediction with Ensemble Learning and Oversampling](https://mlsanity.com/paper/1805.10267/) (2018)

## Built on by (most cited first, 18 of 18)

- [Offensive Language Identification in Low-resourced Code-mixed Dravidian languages using Pseudo-labeling](https://mlsanity.com/paper/2108.12177/) (2021)
- [LLM-Based Multi-Task Bangla Hate Speech Detection: Type, Severity, and Target](https://mlsanity.com/paper/2510.01995/) (2025)
- [Beyond Monolingual Assumptions: A Survey of Code-Switched NLP in the Era of Large Language Models across Modalities](https://mlsanity.com/paper/2510.07037/) (2025)
- [Code-mixed Sentiment and Hate-speech Prediction](https://mlsanity.com/paper/2405.12929/) (2024)
- [The Skipped Beat: A Study of Sociopragmatic Understanding in LLMs for 64 Languages](https://mlsanity.com/paper/2310.14557/) (2023)
- [Findings of the Shared Task on Offensive Span Identification from Code-Mixed Tamil-English Comments](https://mlsanity.com/paper/2205.06118/) (2022)
- [Cross-lingual Offensive Language Detection: A Systematic Review of Datasets, Transfer Approaches and Challenges](https://mlsanity.com/paper/2401.09244/) (2024)
- [Pretrained Transformers for Offensive Language Identification in Tanglish](https://mlsanity.com/paper/2110.02852/) (2021)
- [Zero-shot Code-Mixed Offensive Span Identification through Rationale Extraction](https://mlsanity.com/paper/2205.06119/) (2022)
- [Progressive Sentiment Analysis for Code-Switched Text Data](https://mlsanity.com/paper/2210.14380/) (2022)
- [Statement-Tuning Enables Efficient Cross-lingual Generalization in Encoder-only Models](https://mlsanity.com/paper/2506.01592/) (2025)
- [NepEMO: A Multi-Label Emotion and Sentiment Analysis on Nepali Reddit with Linguistic Insights and Temporal Trends](https://mlsanity.com/paper/2512.22823/) (2025)
- [Hypers at ComMA@ICON: Modelling Aggressiveness, Gender Bias and Communal Bias Identification](https://mlsanity.com/paper/2112.15417/) (2021)
- [TamilEmo: Finegrained Emotion Detection Dataset for Tamil](https://mlsanity.com/paper/2202.04725/) (2022)
- [YouTube Comments Decoded: Leveraging LLMs for Low Resource Language Classification](https://mlsanity.com/paper/2411.05039/) (2024)
- [Creating and Evaluating Code-Mixed Nepali-English and Telugu-English Datasets for Abusive Language Detection Using Traditional and Deep Learning Models](https://mlsanity.com/paper/2504.21026/) (2025)
- [Overcoming Low-Resource Barriers in Tulu: Neural Models and Corpus Creation for OffensiveLanguage Identification](https://mlsanity.com/paper/2508.11166/) (2025)
- [EnTaCs: Analyzing the Relationship Between Sentiment and Language Choice in English-Tamil Code-Switching](https://mlsanity.com/paper/2603.26587/) (2026)

## Abstract (arXiv)

This paper describes the development of a multilingual, manually annotated dataset for three under-resourced Dravidian languages generated from social media comments. The dataset was annotated for sentiment analysis and offensive language identification for a total of more than 60,000 YouTube comments. The dataset consists of around 44,000 comments in Tamil-English, around 7,000 comments in Kannada-English, and around 20,000 comments in Malayalam-English. The data was manually annotated by volunteer annotators and has a high inter-annotator agreement in Krippendorff's alpha. The dataset contains all types of code-mixing phenomena since it comprises user-generated content from a multilingual country. We also present baseline experiments to establish benchmarks on the dataset using machine learning methods. The dataset is available on Github (https://github.com/bharathichezhiyan/DravidianCodeMix-Dataset) and Zenodo (https://zenodo.org/record/4750858#.YJtw0SYo0M).