# Detection of Emotions in Hindi-English Code Mixed Text Data

arXiv: https://arxiv.org/abs/2105.09226 · HTML: https://mlsanity.com/paper/2105.09226/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 26 references that are in the map.
- First posted on arXiv 19 May 2021.

## Builds on (most cited first, 12 of 12)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Enriching Word Vectors with Subword Information](https://mlsanity.com/paper/1607.04606/) (2016)
- [Convolutional Neural Networks for Sentence Classification](https://mlsanity.com/paper/1408.5882/) (2014)
- [Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels](https://mlsanity.com/paper/1805.07836/) (2018)
- [GLUECoS : An Evaluation Benchmark for Code-Switched NLP](https://mlsanity.com/paper/2004.12376/) (2020)
- [A Survey of Code-switched Speech and Language Processing](https://mlsanity.com/paper/1904.00784/) (2019)
- [SemEval-2020 Task 9: Overview of Sentiment Analysis of Code-Mixed Tweets](https://mlsanity.com/paper/2008.04277/) (2020)
- [Processing South Asian Languages Written in the Latin Script: the Dakshina Dataset](https://mlsanity.com/paper/2007.01176/) (2020)
- [Towards Sub-Word Level Compositions for Sentiment Analysis of Hindi-English Code Mixed Text](https://mlsanity.com/paper/1611.00472/) (2016)

## Abstract (arXiv)

Hindi-English code-mixing, the alternation between the two languages within a single utterance, accounts for a substantial share of user-generated content in India. Because such text is written in the Latin script without a standardised transliteration convention, one underlying word surfaces in many spellings and many tokens fall outside English lexica. This paper addresses emotion detection in this setting as a four-way classification problem over anger, fear, happiness and sadness, a finer-grained task than the polarity classification targeted by most prior work. We make three contributions. First, we construct a corpus of 1,589 code-mixed sentences drawn from Twitter and from video-platform comment sections, annotated by two bilingual speakers with an inter-annotator agreement of kappa = 0.94; the video-comment portion covers a longer, less hashtag-dominated register than the Twitter-only corpora used previously. Second, we propose a normalisation procedure that clusters transliteration variants by combining distributional similarity over skip-gram vectors with a hard constraint on consonant identity, since variation is carried almost entirely by vowels. Third, we compare five baselines under a common five-fold cross-validation protocol: Naive Bayes over character and over word n-grams, a word-level LSTM, a sub-word LSTM with convolved character embeddings, and an SVM over frequency-based word vectors. The sub-word LSTM attains the highest accuracy, 76.6%, against a majority-class floor of 30.8%, and the controlled comparison against an otherwise identical word-level LSTM isolates sub-word representation as the source of that gain. On macro-averaged F1, however, it ties with word n-gram Naive Bayes at 0.77, a margin that five-fold cross-validation on 1,589 instances cannot resolve. We report a per-class error analysis and discuss the limitations imposed by corpus scale.