# Challenges and Limitations with the Metrics Measuring the Complexity of Code-Mixed Text

arXiv: https://arxiv.org/abs/2106.10123 · HTML: https://mlsanity.com/paper/2106.10123/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 26 references that are in the map.
- First posted on arXiv 18 Jun 2021.

## Builds on (most cited first, 5 of 5)

- [SemEval-2020 Task 9: Overview of Sentiment Analysis of Code-Mixed Tweets](https://mlsanity.com/paper/2008.04277/) (2020)
- [PHINC: A Parallel Hinglish Social Media Code-Mixed Corpus for Machine Translation](https://mlsanity.com/paper/2004.09447/) (2020)
- [Towards Sub-Word Level Compositions for Sentiment Analysis of Hindi-English Code Mixed Text](https://mlsanity.com/paper/1611.00472/) (2016)
- [A New Dataset for Natural Language Inference from Code-mixed Conversations](https://mlsanity.com/paper/2004.05051/) (2020)
- [A Corpus of English-Hindi Code-Mixed Tweets for Sarcasm Detection](https://mlsanity.com/paper/1805.11869/) (2018)

## Built on by (most cited first, 18 of 18)

- [The Decades Progress on Code-Switching Research in NLP: A Systematic Survey on Trends and Challenges](https://mlsanity.com/paper/2212.09660/) (2022)
- [HinGE: A Dataset for Generation and Evaluation of Code-Mixed Hinglish Text](https://mlsanity.com/paper/2107.03760/) (2021)
- [MIPE: A Metric Independent Pipeline for Effective Code-Mixed NLG Evaluation](https://mlsanity.com/paper/2107.11534/) (2021)
- [Quality Evaluation of the Low-Resource Synthetically Generated Code-Mixed Hinglish Text](https://mlsanity.com/paper/2108.01861/) (2021)
- [COMI-LINGUA: Expert Annotated Large-Scale Dataset for Multitask NLP in Hindi-English Code-Mixing](https://mlsanity.com/paper/2503.21670/) (2025)
- [Can Code-Switched Texts Activate a Knowledge Switch in LLMs? A Case Study on English-Korean Code-Switching](https://mlsanity.com/paper/2410.18436/) (2024)
- [CHAI for LLMs: Improving Code-Mixed Translation in Large Language Models through Reinforcement Learning with AI Feedback](https://mlsanity.com/paper/2411.09073/) (2024)
- [Beyond Monolingual Assumptions: A Survey of Code-Switched NLP in the Era of Large Language Models across Modalities](https://mlsanity.com/paper/2510.07037/) (2025)
- [MMT: A Multilingual and Multi-Topic Indian Social Media Dataset](https://mlsanity.com/paper/2304.00634/) (2023)
- [Conditioning LLMs to Generate Code-Switched Text](https://mlsanity.com/paper/2502.12924/) (2025)
- [MUTANT: A Multi-sentential Code-mixed Hinglish Dataset](https://mlsanity.com/paper/2302.11766/) (2023)
- [From Construction to Injection: Edit-Based Fingerprints for Large Language Models](https://mlsanity.com/paper/2509.03122/) (2025)
- [VietMix: A Naturally-Occurring Parallel Corpus and Augmentation Framework for Vietnamese-English Code-Mixed Machine Translation](https://mlsanity.com/paper/2505.24472/) (2025)
- [LuxBorrow: From Pompier to Pompjee, Tracing Borrowing in Luxembourgish](https://mlsanity.com/paper/2603.10789/) (2026)
- [Improving Code-Switching ASR with Code-Mixing Guided Synthetic Speech](https://mlsanity.com/paper/2606.19381/) (2026)
- [Code Mixologist : A Practitioner's Guide to Building Code-Mixed LLMs](https://mlsanity.com/paper/2602.11181/) (2026)
- [Indi-RomCoM: Code-Mixed Benchmark for Evaluating LLMs on Romanized Indic-English Instructions](https://mlsanity.com/paper/2606.30790/) (2026)
- [AfriSwitch: A Benchmark for In-the-Wild African Code-Switched Speech Recognition](https://mlsanity.com/paper/2608.26434/) (2026)

## Abstract (arXiv)

Code-mixing is a frequent communication style among multilingual speakers where they mix words and phrases from two different languages in the same utterance of text or speech. Identifying and filtering code-mixed text is a challenging task due to its co-existence with monolingual and noisy text. Over the years, several code-mixing metrics have been extensively used to identify and validate code-mixed text quality. This paper demonstrates several inherent limitations of code-mixing metrics with examples from the already existing datasets that are popularly used across various experiments.