# Code-mixed Sentiment and Hate-speech Prediction

arXiv: https://arxiv.org/abs/2405.12929 · HTML: https://mlsanity.com/paper/2405.12929/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 59 references that are in the map.
- First posted on arXiv 21 May 2024.

## Builds on (most cited first, 30 of 34)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [The Power of Scale for Parameter-Efficient Prompt Tuning](https://mlsanity.com/paper/2104.08691/) (2021)
- [QLoRA: Efficient Finetuning of Quantized LLMs](https://mlsanity.com/paper/2305.14314/) (2023)
- [CamemBERT: a Tasty French Language Model](https://mlsanity.com/paper/1911.03894/) (2019)
- [MuRIL: Multilingual Representations for Indian Languages](https://mlsanity.com/paper/2103.10730/) (2021)
- [A Monolingual Approach to Contextualized Word Embeddings for Mid-Resource Languages](https://mlsanity.com/paper/2006.06202/) (2020)
- [Adaptation of Deep Bidirectional Multilingual Transformers for Russian Language](https://mlsanity.com/paper/1905.07213/) (2019)
- [GLUECoS : An Evaluation Benchmark for Code-Switched NLP](https://mlsanity.com/paper/2004.12376/) (2020)
- [Multilingual Large Language Models Are Not (Yet) Code-Switchers](https://mlsanity.com/paper/2305.14235/) (2023)
- [Are Multilingual Models Effective in Code-Switching?](https://mlsanity.com/paper/2103.13309/) (2021)
- [TwHIN-BERT: A Socially-Enriched Pre-trained Language Model for Multilingual Tweet Representations at Twitter](https://mlsanity.com/paper/2209.07562/) (2022)
- [PHINC: A Parallel Hinglish Social Media Code-Mixed Corpus for Machine Translation](https://mlsanity.com/paper/2004.09447/) (2020)
- [L3Cube-HindBERT and DevBERT: Pre-Trained BERT Transformer models for Devanagari based Hindi and Marathi Languages](https://mlsanity.com/paper/2211.11418/) (2022)
- [L3Cube-HingCorpus and HingBERT: A Code Mixed Hindi-English Dataset and BERT Language Models](https://mlsanity.com/paper/2204.08398/) (2022)
- [Multi-modal Sarcasm Detection and Humor Classification in Code-mixed Conversations](https://mlsanity.com/paper/2105.09984/) (2021)
- [Towards Sub-Word Level Compositions for Sentiment Analysis of Hindi-English Code Mixed Text](https://mlsanity.com/paper/1611.00472/) (2016)
- [A New Dataset for Natural Language Inference from Code-mixed Conversations](https://mlsanity.com/paper/2004.05051/) (2020)
- [Named Entity Recognition on Code-Switched Data: Overview of the CALCS 2018 Shared Task](https://mlsanity.com/paper/1906.04138/) (2019)
- [Code-Mixing on Sesame Street: Dawn of the Adversarial Polyglots](https://mlsanity.com/paper/2103.09593/) (2021)
- [FinEst BERT and CroSloEngual BERT: less is more in multilingual models](https://mlsanity.com/paper/2006.07890/) (2020)
- [DravidianCodeMix: Sentiment Analysis and Offensive Language Identification Dataset for Dravidian Languages in Code-Mixed Text](https://mlsanity.com/paper/2106.09460/) (2021)
- [BERTweetFR : Domain Adaptation of Pre-Trained Language Models for French Tweets](https://mlsanity.com/paper/2109.10234/) (2021)
- [NLP-CUET@LT-EDI-EACL2021: Multilingual Code-Mixed Hope Speech Detection using Cross-lingual Representation Learner](https://mlsanity.com/paper/2103.00464/) (2021)
- [Training dataset and dictionary sizes matter in BERT models: the case of Baltic languages](https://mlsanity.com/paper/2112.10553/) (2021)
- [How to evaluate sentiment classifiers for Twitter time-ordered data?](https://mlsanity.com/paper/1803.05160/) (2018)

## Built on by (most cited first, 4 of 4)

- [Beyond Monolingual Assumptions: A Survey of Code-Switched NLP in the Era of Large Language Models across Modalities](https://mlsanity.com/paper/2510.07037/) (2025)
- [What talking you?: Translating Code-Mixed Messaging Texts to English](https://mlsanity.com/paper/2411.05253/) (2024)
- [Gen-AI for User Safety: A Survey](https://mlsanity.com/paper/2411.06606/) (2024)
- [Rethinking Hate Speech Detection on Social Media: Can LLMs Replace Traditional Models?](https://mlsanity.com/paper/2506.12744/) (2025)

## Abstract (arXiv)

Code-mixed discourse combines multiple languages in a single text. It is commonly used in informal discourse in countries with several official languages, but also in many other countries in combination with English or neighboring languages. As recently large language models have dominated most natural language processing tasks, we investigated their performance in code-mixed settings for relevant tasks. We first created four new bilingual pre-trained masked language models for English-Hindi and English-Slovene languages, specifically aimed to support informal language. Then we performed an evaluation of monolingual, bilingual, few-lingual, and massively multilingual models on several languages, using two tasks that frequently contain code-mixed text, in particular, sentiment analysis and offensive language detection in social media texts. The results show that the most successful classifiers are fine-tuned bilingual models and multilingual models, specialized for social media texts, followed by non-specialized massively multilingual and monolingual models, while huge generative models are not competitive. For our affective problems, the models mostly perform slightly better on code-mixed data compared to non-code-mixed data.