# COMI-LINGUA: Expert Annotated Large-Scale Dataset for Multitask NLP in Hindi-English Code-Mixing

arXiv: https://arxiv.org/abs/2503.21670 · HTML: https://mlsanity.com/paper/2503.21670/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 74 references that are in the map.
- First posted on arXiv 27 Mar 2025.

## Builds on (most cited first, 23 of 23)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [GLUECoS : An Evaluation Benchmark for Code-Switched NLP](https://mlsanity.com/paper/2004.12376/) (2020)
- [LinCE: A Centralized Benchmark for Linguistic Code-switching Evaluation](https://mlsanity.com/paper/2005.04322/) (2020)
- [The IIT Bombay English-Hindi Parallel Corpus](https://mlsanity.com/paper/1710.02855/) (2017)
- [Aggression-annotated Corpus of Hindi-English Code-mixed Data](https://mlsanity.com/paper/1803.09402/) (2018)
- [PHINC: A Parallel Hinglish Social Media Code-Mixed Corpus for Machine Translation](https://mlsanity.com/paper/2004.09447/) (2020)
- [L3Cube-HingCorpus and HingBERT: A Code Mixed Hindi-English Dataset and BERT Language Models](https://mlsanity.com/paper/2204.08398/) (2022)
- [Overview for the Second Shared Task on Language Identification in Code-Switched Data](https://mlsanity.com/paper/1909.13016/) (2019)
- [A New Dataset for Natural Language Inference from Code-mixed Conversations](https://mlsanity.com/paper/2004.05051/) (2020)
- [HinGE: A Dataset for Generation and Evaluation of Code-Mixed Hinglish Text](https://mlsanity.com/paper/2107.03760/) (2021)
- [Challenges and Limitations with the Metrics Measuring the Complexity of Code-Mixed Text](https://mlsanity.com/paper/2106.10123/) (2021)
- [A Corpus of English-Hindi Code-Mixed Tweets for Sarcasm Detection](https://mlsanity.com/paper/1805.11869/) (2018)
- [HiNER: A Large Hindi Named Entity Recognition Dataset](https://mlsanity.com/paper/2204.13743/) (2022)
- [Word Level Language Identification in English Telugu Code Mixed Data](https://mlsanity.com/paper/2010.04482/) (2020)
- [Quality Evaluation of the Low-Resource Synthetically Generated Code-Mixed Hinglish Text](https://mlsanity.com/paper/2108.01861/) (2021)
- [Evaluating Input Representation for Language Identification in Hindi-English Code Mixed Text](https://mlsanity.com/paper/2011.11263/) (2020)
- [Synthetic Data Generation and Joint Learning for Robust Code-Mixed Translation](https://mlsanity.com/paper/2403.16771/) (2024)
- [CMNEROne at SemEval-2022 Task 11: Code-Mixed Named Entity Recognition by leveraging multilingual data](https://mlsanity.com/paper/2206.07318/) (2022)
- [Leveraging Large Language Models for Code-Mixed Data Augmentation in Sentiment Analysis](https://mlsanity.com/paper/2411.00691/) (2024)
- [Cross Script Hindi English NER Corpus from Wikipedia](https://mlsanity.com/paper/1810.03430/) (2018)
- [MUTANT: A Multi-sentential Code-mixed Hinglish Dataset](https://mlsanity.com/paper/2302.11766/) (2023)

## Built on by (most cited first, 7 of 7)

- [Beyond Monolingual Assumptions: A Survey of Code-Switched NLP in the Era of Large Language Models across Modalities](https://mlsanity.com/paper/2510.07037/) (2025)
- [Script Gap: Evaluating LLM Triage on Indian Languages in Native vs Romanized Scripts in a Real World Setting](https://mlsanity.com/paper/2512.10780/) (2025)
- [VietMix: A Naturally-Occurring Parallel Corpus and Augmentation Framework for Vietnamese-English Code-Mixed Machine Translation](https://mlsanity.com/paper/2505.24472/) (2025)
- [BhashaSutra: A Task-Centric Unified Survey of Indian NLP Datasets, Corpora, and Resources](https://mlsanity.com/paper/2604.18423/) (2026)
- [Code Mixologist : A Practitioner's Guide to Building Code-Mixed LLMs](https://mlsanity.com/paper/2602.11181/) (2026)
- [PERCEPT: A Corpus for POS Tagging and Analysis of Persian-English Code-Mixing](https://mlsanity.com/paper/2608.10109/) (2026)
- [TurEngMix: A Text Corpus and Benchmark for Turkish-English Code-Mixed Language Identification and Named Entity Recognition](https://mlsanity.com/paper/2609.06963/) (2026)

## Abstract (arXiv)

We introduce COMI-LINGUA, the largest manually annotated Hindi-English code-mixed dataset, comprising 125K+ high-quality instances across five core NLP tasks: Matrix Language Identification, Token-level Language Identification, Part-Of-Speech Tagging, Named Entity Recognition, and Machine Translation. Each instance is annotated by three bilingual annotators, yielding over 376K expert annotations with strong inter-annotator agreement (Fleiss' Kappa ≥ 0.81). The rigorously preprocessed and filtered dataset covers both Devanagari and Roman scripts and spans diverse domains, ensuring real-world linguistic coverage. Evaluation reveals that closed-source LLMs significantly outperform traditional tools and open-source models in zero-shot settings. Notably, one-shot prompting consistently boosts performance across tasks, especially in structure-sensitive predictions like POS and NER. Fine-tuning state-of-the-art LLMs on COMI-LINGUA demonstrates substantial improvements, achieving up to 95.25 F1 in NER, 98.77 F1 in MLI, and competitive MT performance, setting new benchmarks for Hinglish code-mixed text. COMI-LINGUA is publicly available at this URL: https://huggingface.co/datasets/LingoIITGN/COMI-LINGUA.