# Quality Evaluation of the Low-Resource Synthetically Generated Code-Mixed Hinglish Text

arXiv: https://arxiv.org/abs/2108.01861 · HTML: https://mlsanity.com/paper/2108.01861/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 24 references that are in the map.
- First posted on arXiv 4 Aug 2021.

## Builds on (most cited first, 10 of 10)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Enriching Word Vectors with Subword Information](https://mlsanity.com/paper/1607.04606/) (2016)
- [The IIT Bombay English-Hindi Parallel Corpus](https://mlsanity.com/paper/1710.02855/) (2017)
- [SemEval-2020 Task 9: Overview of Sentiment Analysis of Code-Mixed Tweets](https://mlsanity.com/paper/2008.04277/) (2020)
- [Are Multilingual Models Effective in Code-Switching?](https://mlsanity.com/paper/2103.13309/) (2021)
- [PHINC: A Parallel Hinglish Social Media Code-Mixed Corpus for Machine Translation](https://mlsanity.com/paper/2004.09447/) (2020)
- [Overview for the Second Shared Task on Language Identification in Code-Switched Data](https://mlsanity.com/paper/1909.13016/) (2019)
- [HinGE: A Dataset for Generation and Evaluation of Code-Mixed Hinglish Text](https://mlsanity.com/paper/2107.03760/) (2021)
- [Challenges and Limitations with the Metrics Measuring the Complexity of Code-Mixed Text](https://mlsanity.com/paper/2106.10123/) (2021)
- [Task-Specific Pre-Training and Cross Lingual Transfer for Code-Switched Data](https://mlsanity.com/paper/2102.12407/) (2021)

## Built on by (most cited first, 7 of 7)

- [The Decades Progress on Code-Switching Research in NLP: A Systematic Survey on Trends and Challenges](https://mlsanity.com/paper/2212.09660/) (2022)
- [From Human Judgements to Predictive Models: Unravelling Acceptability in Code-Mixed Sentences](https://mlsanity.com/paper/2405.05572/) (2024)
- [COMI-LINGUA: Expert Annotated Large-Scale Dataset for Multitask NLP in Hindi-English Code-Mixing](https://mlsanity.com/paper/2503.21670/) (2025)
- [niksss at HinglishEval: Language-agnostic BERT-based Contextual Embeddings with Catboost for Quality Evaluation of the Low-Resource Synthetically Generated Code-Mixed Hinglish Text](https://mlsanity.com/paper/2206.08910/) (2022)
- [PreCogIIITH at HinglishEval : Leveraging Code-Mixing Metrics & Language Model Embeddings To Estimate Code-Mix Quality](https://mlsanity.com/paper/2206.07988/) (2022)
- [JU_NLP at HinglishEval: Quality Evaluation of the Low-Resource Code-Mixed Hinglish Text](https://mlsanity.com/paper/2206.08053/) (2022)
- [BITS Pilani at HinglishEval: Quality Evaluation for Code-Mixed Hinglish Text Using Transformers](https://mlsanity.com/paper/2206.08680/) (2022)

## Abstract (arXiv)

In this shared task, we seek the participating teams to investigate the factors influencing the quality of the code-mixed text generation systems. We synthetically generate code-mixed Hinglish sentences using two distinct approaches and employ human annotators to rate the generation quality. We propose two subtasks, quality rating prediction and annotators' disagreement prediction of the synthetic Hinglish dataset. The proposed subtasks will put forward the reasoning and explanation of the factors influencing the quality and human perception of the code-mixed text.