SemEval-2020 Task 9: Overview of Sentiment Analysis of Code-Mixed Tweets
Not on the map. Why?
Abstract (arXiv)
In this paper, we present the results of the SemEval-2020 Task 9 on Sentiment Analysis of Code-Mixed Tweets (SentiMix 2020). We also release and describe our Hinglish (Hindi-English) and Spanglish (Spanish-English) corpora annotated with word-level language identification and sentence-level sentiment labels. These corpora are comprised of 20K and 19K examples, respectively. The sentiment labels are - Positive, Negative, and Neutral. SentiMix attracted 89 submissions in total including 61 teams that participated in the Hinglish contest and 28 submitted systems to the Spanglish competition. The best performance achieved was 75.0% F1 score for Hinglish and 80.6% F1 for Spanglish. We observe that BERT-like models and ensemble methods are the most common and successful approaches among the participants.
Key facts. Cited by 62 papers in the map (data through 7 Oct 2026). First posted on arXiv 10 Aug 2020.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 23; 2022: 9; 2023: 9; 2024: 9; 2025: 9; 2026: 3.
62 papers.
Sort:
By area:
- 2021XLM-T: Multilingual Language Models in Twitter for Sentiment Analysis and Beyondcited by 114
- 2022TwHIN-BERT: A Socially-Enriched Pre-trained Language Model for Multilingual Tweet Representations at Twittercited by 44
- 2023SemEval-2023 Task 12: Sentiment Analysis for African Languages (AfriSenti-SemEval)cited by 41
- 2022The Decades Progress on Code-Switching Research in NLP: A Systematic Survey on Trends and Challengescited by 35
- 2021HinGE: A Dataset for Generation and Evaluation of Code-Mixed Hinglish Textcited by 28
- 2022PQLM -- Multilingual Decentralized Portable Quantum Language Model for Privacy Protectioncited by 26
- 2021Code-Mixing on Sesame Street: Dawn of the Adversarial Polyglotscited by 23
- 2021RoBERTuito: a pre-trained language model for social media text in Spanishcited by 22
- 2021Can You Traducir This? Machine Translation for Code-Switched Inputcited by 21
- 2021DravidianCodeMix: Sentiment Analysis and Offensive Language Identification Dataset for Dravidian Languages in Code-Mixed Textcited by 18
- 2021Challenges and Limitations with the Metrics Measuring the Complexity of Code-Mixed Textcited by 18
- 2021HeBERT & HebEMO: a Hebrew BERT Model and a Tool for Polarity Analysis and Emotion Recognitioncited by 14
38 more
- 2023Memotion 3: Dataset on Sentiment and Emotion Analysis of Codemixed Hindi-English Memescited by 12
- 2021SJ_AJ@DravidianLangTech-EACL2021: Task-Adaptive Pre-Training of Multilingual BERT models for Offensive Language Identificationcited by 11
- 2024MINERS: Multilingual Language Models as Semantic Retrieverscited by 11
- 2023Massively Multilingual Corpus of Sentiment Datasets and Multi-faceted Sentiment Classification Benchmarkcited by 10
- 2025Lost in the Mix: Evaluating LLM Understanding of Code-Switched Textcited by 9
- 2021CodemixedNLP: An Extensible and Open NLP Toolkit for Code-Mixingcited by 8
- 2025CodeMixBench: Evaluating Code-Mixing Capabilities of LLMs Across 18 Languagescited by 8
- 2021Unsupervised Self-Training for Sentiment Analysis of Code-Switched Datacited by 7
- 2021MIPE: A Metric Independent Pipeline for Effective Code-Mixed NLG Evaluationcited by 7
- 2021Quality Evaluation of the Low-Resource Synthetically Generated Code-Mixed Hinglish Textcited by 7
- 2024Code-Mixed Probes Show How Pre-Trained Models Generalise On Code-Switched Textcited by 7
- 2021Sentiment Analysis of Persian-English Code-mixed Textscited by 6
- 2021Offense Detection in Dravidian Languages using Code-Mixing Index based Focal Losscited by 6
- 2021Task-Specific Pre-Training and Cross Lingual Transfer for Code-Switched Datacited by 5
- 2024Synthetic Data Generation and Joint Learning for Robust Code-Mixed Translationcited by 5
- 2024CHAI for LLMs: Improving Code-Mixed Translation in Large Language Models through Reinforcement Learning with AI Feedbackcited by 5
- 2021Challenges and Considerations with Code-Mixed NLP for Multilingual Societiescited by 4
- 2021The Effectiveness of Intermediate-Task Training for Code-Switched Natural Language Understandingcited by 4
- 2025Beyond Monolingual Assumptions: A Survey of Code-Switched NLP in the Era of Large Language Models across Modalitiescited by 4
- 2021CMSAOne@Dravidian-CodeMix-FIRE2020: A Meta Embedding and Transformer model for Code-Mixed Sentiment Analysis on Social Media Textcited by 3
- 2021AdvCodeMix: Adversarial Attack on Code-Mixed Datacited by 3
- 2021PESTO: Switching Point based Dynamic and Relative Positional Encoding for Code-Mixed Languagescited by 3
- 2022Negativity Spreads Faster: A Large-Scale Multilingual Twitter Analysis on the Role of Sentiment in Political Communicationcited by 3
- 2022Assessment of Massively Multilingual Sentiment Classifierscited by 3
- 2022Language Agnostic Code-Mixing Data Augmentation by Predicting Linguistic Patternscited by 3
- 2024Leveraging Large Language Models for Code-Mixed Data Augmentation in Sentiment Analysiscited by 3
- 2025Low-resource Machine Translation for Code-switched Kazakh-Russian Language Paircited by 3
- 2025DIVERS-Bench: Evaluating Language Identification Across Domain Shifts and Code-Switchingcited by 3
- 2021One Source, Two Targets: Challenges and Rewards of Dual Decodingcited by 2
- 2022A Comprehensive Understanding of Code-mixed Language Semantics using Hierarchical Transformercited by 2
- 2023Bangla Grammatical Error Detection Using T5 Transformer Modelcited by 2
- 2023CONFLATOR: Incorporating Switching Point based Rotatory Positional Encodings for Code-Mixed Language Modelingcited by 2
- 2023Overview of Memotion 3: Sentiment and Emotion Analysis of Codemixed Hinglish Memescited by 2
- 2025Conditioning LLMs to Generate Code-Switched Textcited by 2
- 2023Shared Lexical Items as Triggers of Code Switchingcited by 1
- 2023The Effect of Alignment Objectives on Code-Switching Translationcited by 1
- 2023Elevating Code-mixed Text Handling through Auditory Information of Wordscited by 1
- 2024Sociolinguistically Informed Interpretability: A Case Study on Hinglish Emotion Classificationcited by 1
50 most cited of 62 All 62 (JSON)
- 2026100,000+ Movie Reviews from Kazakhstan: Russian, Kazakh, and Code-Switched Texts9 May 2026
- 2026Neither Here Nor There: Cross-Lingual Representation Dynamics of Code-Mixed Text in Multilingual Encoders20 Mar 2026
- 2026NLP Privacy Risk Identification in Social Media (NLP-PRISM): A Survey26 Jan 2026
- 2025Modeling Topics and Sociolinguistic Variation in Code-Switched Discourse: Insights from Spanish-English and Spanish-Guaran'i3 Dec 2025
- 2025Adapting Multilingual Models to Code-Mixed Tasks via Model Merging22 Oct 2025
- 2025Beyond Monolingual Assumptions: A Survey of Code-Switched NLP in the Era of Large Language Models across Modalities8 Oct 2025
- 2025DIVERS-Bench: Evaluating Language Identification Across Domain Shifts and Code-Switching22 Sep 2025
- 2025Human-Centered Supervision for Sentiment Analysis in Telugu: A Systematic Inquiry Beyond Accuracy2 Aug 2025
- 2025CodeMixBench: Evaluating Code-Mixing Capabilities of LLMs Across 18 Languages24 Jul 2025
- 2025Lost in the Mix: Evaluating LLM Understanding of Code-Switched Text16 Jun 2025
- 2025Low-resource Machine Translation for Code-switched Kazakh-Russian Language Pair25 Mar 2025
- 2025Conditioning LLMs to Generate Code-Switched Text18 Feb 2025
38 more
- 2024CHAI for LLMs: Improving Code-Mixed Translation in Large Language Models through Reinforcement Learning with AI Feedback13 Nov 2024
- 2024Leveraging Large Language Models for Code-Mixed Data Augmentation in Sentiment Analysis1 Nov 2024
- 2024MINERS: Multilingual Language Models as Semantic Retrievers11 Jun 2024
- 2024Synthetic Data Generation and Joint Learning for Robust Code-Mixed Translation25 Mar 2024
- 2024Code-Mixed Probes Show How Pre-Trained Models Generalise On Code-Switched Text7 Mar 2024
- 2024MaCmS: Magahi Code-mixed Dataset for Sentiment Analysis7 Mar 2024
- 2024Improving Cross-lingual Representation for Semantic Retrieval with Code-switching3 Mar 2024
- 2024Share What You Already Know: Cross-Language-Script Transfer and Alignment for Sentiment Detection in Code-Mixed Data7 Feb 2024
- 2024Sociolinguistically Informed Interpretability: A Case Study on Hinglish Emotion Classification5 Feb 2024
- 2023Elevating Code-mixed Text Handling through Auditory Information of Words27 Oct 2023
- 2023Overview of Memotion 3: Sentiment and Emotion Analysis of Codemixed Hinglish Memes12 Sep 2023
- 2023CONFLATOR: Incorporating Switching Point based Rotatory Positional Encodings for Code-Mixed Language Modeling11 Sep 2023
- 2023The Effect of Alignment Objectives on Code-Switching Translation10 Sep 2023
- 2023Shared Lexical Items as Triggers of Code Switching29 Aug 2023
- 2023Massively Multilingual Corpus of Sentiment Datasets and Multi-faceted Sentiment Classification Benchmark13 Jun 2023
- 2023SemEval-2023 Task 12: Sentiment Analysis for African Languages (AfriSenti-SemEval)13 Apr 2023
- 2023Bangla Grammatical Error Detection Using T5 Transformer Model19 Mar 2023
- 2023Memotion 3: Dataset on Sentiment and Emotion Analysis of Codemixed Hindi-English Memes17 Mar 2023
- 2022The Decades Progress on Code-Switching Research in NLP: A Systematic Survey on Trends and Challenges19 Dec 2022
- 2022Language Agnostic Code-Mixing Data Augmentation by Predicting Linguistic Patterns14 Nov 2022
- 2022Sentiment Classification of Code-Switched Text using Pre-trained Multilingual Embeddings and Segmentation29 Oct 2022
- 2022PQLM -- Multilingual Decentralized Portable Quantum Language Model for Privacy Protection6 Oct 2022
- 2022TwHIN-BERT: A Socially-Enriched Pre-trained Language Model for Multilingual Tweet Representations at Twitter15 Sep 2022
- 2022A Comprehensive Understanding of Code-mixed Language Semantics using Hierarchical Transformer27 Apr 2022
- 2022Assessment of Massively Multilingual Sentiment Classifiers11 Apr 2022
- 2022Multilingual Abusiveness Identification on Code-Mixed Social Media Text1 Mar 2022
- 2022Negativity Spreads Faster: A Large-Scale Multilingual Twitter Analysis on the Role of Sentiment in Political Communication1 Feb 2022
- 2021RoBERTuito: a pre-trained language model for social media text in Spanish18 Nov 2021
- 2021Offense Detection in Dravidian Languages using Code-Mixing Index based Focal Loss12 Nov 2021
- 2021PESTO: Switching Point based Dynamic and Relative Positional Encoding for Code-Mixed Languages12 Nov 2021
- 2021AdvCodeMix: Adversarial Attack on Code-Mixed Data30 Oct 2021
- 2021One Source, Two Targets: Challenges and Rewards of Dual Decoding21 Sep 2021
- 2021Quality Evaluation of the Low-Resource Synthetically Generated Code-Mixed Hinglish Text4 Aug 2021
- 2021MIPE: A Metric Independent Pipeline for Effective Code-Mixed NLG Evaluation24 Jul 2021
- 2021The Effectiveness of Intermediate-Task Training for Code-Switched Natural Language Understanding21 Jul 2021
- 2021HinGE: A Dataset for Generation and Evaluation of Code-Mixed Hinglish Text8 Jul 2021
- 2021Challenges and Limitations with the Metrics Measuring the Complexity of Code-Mixed Text18 Jun 2021
- 2021DravidianCodeMix: Sentiment Analysis and Offensive Language Identification Dataset for Dravidian Languages in Code-Mixed Text17 Jun 2021
50 newest of 62 All 62 (JSON)
- 2021XLM-T: Multilingual Language Models in Twitter for Sentiment Analysis and Beyondcited by 114
- 2022TwHIN-BERT: A Socially-Enriched Pre-trained Language Model for Multilingual Tweet Representations at Twittercited by 44
- 2023SemEval-2023 Task 12: Sentiment Analysis for African Languages (AfriSenti-SemEval)cited by 41
- 2022The Decades Progress on Code-Switching Research in NLP: A Systematic Survey on Trends and Challengescited by 35
- 2021HinGE: A Dataset for Generation and Evaluation of Code-Mixed Hinglish Textcited by 28
- 2022PQLM -- Multilingual Decentralized Portable Quantum Language Model for Privacy Protectioncited by 26
- 2021Code-Mixing on Sesame Street: Dawn of the Adversarial Polyglotscited by 23
- 2021RoBERTuito: a pre-trained language model for social media text in Spanishcited by 22
- 2021Can You Traducir This? Machine Translation for Code-Switched Inputcited by 21
- 2021DravidianCodeMix: Sentiment Analysis and Offensive Language Identification Dataset for Dravidian Languages in Code-Mixed Textcited by 18
- 2021Challenges and Limitations with the Metrics Measuring the Complexity of Code-Mixed Textcited by 18
- 2021HeBERT & HebEMO: a Hebrew BERT Model and a Tool for Polarity Analysis and Emotion Recognitioncited by 14
12 most cited of 61 All 61 (JSON)
- 2026100,000+ Movie Reviews from Kazakhstan: Russian, Kazakh, and Code-Switched Texts9 May 2026
- 2026Neither Here Nor There: Cross-Lingual Representation Dynamics of Code-Mixed Text in Multilingual Encoders20 Mar 2026
- 2026NLP Privacy Risk Identification in Social Media (NLP-PRISM): A Survey26 Jan 2026
- 2025Modeling Topics and Sociolinguistic Variation in Code-Switched Discourse: Insights from Spanish-English and Spanish-Guaran'i3 Dec 2025
- 2025Adapting Multilingual Models to Code-Mixed Tasks via Model Merging22 Oct 2025
- 2025Beyond Monolingual Assumptions: A Survey of Code-Switched NLP in the Era of Large Language Models across Modalities8 Oct 2025
- 2025DIVERS-Bench: Evaluating Language Identification Across Domain Shifts and Code-Switching22 Sep 2025
- 2025Human-Centered Supervision for Sentiment Analysis in Telugu: A Systematic Inquiry Beyond Accuracy2 Aug 2025
- 2025CodeMixBench: Evaluating Code-Mixing Capabilities of LLMs Across 18 Languages24 Jul 2025
- 2025Lost in the Mix: Evaluating LLM Understanding of Code-Switched Text16 Jun 2025
- 2025Low-resource Machine Translation for Code-switched Kazakh-Russian Language Pair25 Mar 2025
- 2025Conditioning LLMs to Generate Code-Switched Text18 Feb 2025
12 newest of 61 All 61 (JSON)