# From Machine Translation to Code-Switching: Generating High-Quality Code-Switched Text

arXiv: https://arxiv.org/abs/2107.06483 · HTML: https://mlsanity.com/paper/2107.06483/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 63 references that are in the map.
- First posted on arXiv 14 Jul 2021.

## Builds on (most cited first, 26 of 26)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Multilingual Denoising Pre-training for Neural Machine Translation](https://mlsanity.com/paper/2001.08210/) (2020)
- [Pointer Networks](https://mlsanity.com/paper/1506.03134/) (2015)
- [Improving Neural Machine Translation Models with Monolingual Data](https://mlsanity.com/paper/1511.06709/) (2015)
- [How multilingual is Multilingual BERT?](https://mlsanity.com/paper/1906.01502/) (2019)
- [Texygen: A Benchmarking Platform for Text Generation Models](https://mlsanity.com/paper/1802.01886/) (2018)
- [Word Translation Without Parallel Data](https://mlsanity.com/paper/1710.04087/) (2017)
- [SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient](https://mlsanity.com/paper/1609.05473/) (2016)
- [Improving Massively Multilingual Neural Machine Translation and Zero-Shot Translation](https://mlsanity.com/paper/2004.11867/) (2020)
- [Unsupervised Machine Translation Using Monolingual Corpora Only](https://mlsanity.com/paper/1711.00043/) (2017)
- [Regularizing and Optimizing LSTM Language Models](https://mlsanity.com/paper/1708.02182/) (2017)
- [SimAlign: High Quality Word Alignments without Parallel Training Data using Static and Contextualized Embeddings](https://mlsanity.com/paper/2004.08728/) (2020)
- [Phrase-Based & Neural Unsupervised Machine Translation](https://mlsanity.com/paper/1804.07755/) (2018)
- [GLUECoS : An Evaluation Benchmark for Code-Switched NLP](https://mlsanity.com/paper/2004.12376/) (2020)
- [The IIT Bombay English-Hindi Parallel Corpus](https://mlsanity.com/paper/1710.02855/) (2017)
- [Code-Switched Language Models Using Neural Based Synthetic Data from Parallel Sentences](https://mlsanity.com/paper/1909.08582/) (2019)
- [Code-switching Sentence Generation by Generative Adversarial Networks and its Application to Data Augmentation](https://mlsanity.com/paper/1811.02356/) (2018)
- [Code-switched Language Models Using Dual RNNs and Same-Source Pretraining](https://mlsanity.com/paper/1809.01962/) (2018)
- [A Deep Generative Model for Code-Switched Text](https://mlsanity.com/paper/1906.08972/) (2019)
- [Grammatical Constraints on Intra-sentential Code-Switching: From Theories to Working Models](https://mlsanity.com/paper/1612.04538/) (2016)
- [Learn to Code-Switch: Data Augmentation using Copy Mechanism on Language Modeling](https://mlsanity.com/paper/1810.10254/) (2018)
- [Syntactic and Semantic Features For Code-Switching Factored Language Models](https://mlsanity.com/paper/1710.01809/) (2017)
- [Dual Language Models for Code Switched Speech Recognition](https://mlsanity.com/paper/1711.01048/) (2017)

## Built on by (most cited first, 29 of 29)

- [Prompting Multilingual Large Language Models to Generate Code-Mixed Texts: The Case of South East Asian Languages](https://mlsanity.com/paper/2303.13592/) (2023)
- [LLM-powered Data Augmentation for Enhanced Cross-lingual Performance](https://mlsanity.com/paper/2305.14288/) (2023)
- [The Decades Progress on Code-Switching Research in NLP: A Systematic Survey on Trends and Challenges](https://mlsanity.com/paper/2212.09660/) (2022)
- [Representativeness as a Forgotten Lesson for Multilingual and Code-switched Data Collection and Preparation](https://mlsanity.com/paper/2310.20470/) (2023)
- [ArzEn-ST: A Three-way Speech Translation Corpus for Code-Switched Egyptian Arabic - English](https://mlsanity.com/paper/2211.12000/) (2022)
- [CALCS 2021 Shared Task: Machine Translation for Code-Switched Data](https://mlsanity.com/paper/2202.09625/) (2022)
- [Investigating Lexical Replacements for Arabic-English Code-Switched Data Augmentation](https://mlsanity.com/paper/2205.12649/) (2022)
- [CroCoSum: A Benchmark Dataset for Cross-Lingual Code-Switched Summarization](https://mlsanity.com/paper/2303.04092/) (2023)
- [Unified model for code-switching speech recognition and language identification based on a concatenated tokenizer](https://mlsanity.com/paper/2306.08753/) (2023)
- [Exploring Segmentation Approaches for Neural Machine Translation of Code-Switched Egyptian Arabic-English Text](https://mlsanity.com/paper/2210.06990/) (2022)
- [Code-Switched Text Synthesis in Unseen Language Pairs](https://mlsanity.com/paper/2305.16724/) (2023)
- [SC-MoE: Switch Conformer Mixture of Experts for Unified Streaming and Non-streaming Code-Switching ASR](https://mlsanity.com/paper/2406.18021/) (2024)
- [Overcoming Vocabulary Constraints with Pixel-level Fallback](https://mlsanity.com/paper/2504.02122/) (2025)
- [Optimizing Bilingual Neural Transducer with Synthetic Code-switching Text Generation](https://mlsanity.com/paper/2210.12214/) (2022)
- [CST5: Data Augmentation for Code-Switched Semantic Parsing](https://mlsanity.com/paper/2211.07514/) (2022)
- [CHAI for LLMs: Improving Code-Mixed Translation in Large Language Models through Reinforcement Learning with AI Feedback](https://mlsanity.com/paper/2411.09073/) (2024)
- [Beyond Monolingual Assumptions: A Survey of Code-Switched NLP in the Era of Large Language Models across Modalities](https://mlsanity.com/paper/2510.07037/) (2025)
- [The Effectiveness of Intermediate-Task Training for Code-Switched Natural Language Understanding](https://mlsanity.com/paper/2107.09931/) (2021)
- [Boosting Zero-shot Cross-lingual Retrieval by Training on Artificially Code-Switched Data](https://mlsanity.com/paper/2305.05295/) (2023)
- [Breaking the Language Barrier: Improving Cross-Lingual Reasoning with Structured Self-Attention](https://mlsanity.com/paper/2310.15258/) (2023)
- [Leveraging Large Language Models for Code-Mixed Data Augmentation in Sentiment Analysis](https://mlsanity.com/paper/2411.00691/) (2024)
- [Data Augmentation Techniques for Machine Translation of Code-Switched Texts: A Comparative Study](https://mlsanity.com/paper/2310.15262/) (2023)
- [Conditioning LLMs to Generate Code-Switched Text](https://mlsanity.com/paper/2502.12924/) (2025)
- [On Creating an English-Thai Code-switched Machine Translation in Medical Domain](https://mlsanity.com/paper/2410.16221/) (2024)
- [CoVoSwitch: Machine Translation of Synthetic Code-Switched Text Based on Intonation Units](https://mlsanity.com/paper/2407.14295/) (2024)
- [The Impact of Code-switched Synthetic Data Quality is Task Dependent: Insights from MT and ASR](https://mlsanity.com/paper/2503.23576/) (2025)
- [Is linguistically-motivated data augmentation worth it?](https://mlsanity.com/paper/2506.03593/) (2025)
- [PingPong: A Natural Benchmark for Multi-Turn Code-Switching Dialogues](https://mlsanity.com/paper/2601.17277/) (2026)
- [From Standard English to Singlish: A Retrieval-Augmented Approach for Code-Switched Creole Generation in Large Language Models](https://mlsanity.com/paper/2605.07132/) (2026)

## Abstract (arXiv)

Generating code-switched text is a problem of growing interest, especially given the scarcity of corpora containing large volumes of real code-switched text. In this work, we adapt a state-of-the-art neural machine translation model to generate Hindi-English code-switched sentences starting from monolingual Hindi sentences. We outline a carefully designed curriculum of pretraining steps, including the use of synthetic code-switched text, that enable the model to generate high-quality code-switched text. Using text generated from our model as data augmentation, we show significant reductions in perplexity on a language modeling task, compared to using text from other generative models of CS text. We also show improvements using our text for a downstream code-switched natural language inference task. Our generated text is further subjected to a rigorous evaluation using a human evaluation study and a range of objective metrics, where we show performance comparable (and sometimes even superior) to code-switched text obtained via crowd workers who are native Hindi speakers.