# Code-Switched Language Models Using Neural Based Synthetic Data from Parallel Sentences

arXiv: https://arxiv.org/abs/1909.08582 · HTML: https://mlsanity.com/paper/1909.08582/

## Key facts

- Cited by 55 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Sep 2019.

## Built on by (most cited first, 30 of 55)

- [Language Models are Few-shot Multilingual Learners](https://mlsanity.com/paper/2109.07684/) (2021)
- [One Country, 700+ Languages: NLP Challenges for Underrepresented Languages and Dialects in Indonesia](https://mlsanity.com/paper/2203.13357/) (2022)
- [Are Multilingual Models Effective in Code-Switching?](https://mlsanity.com/paper/2103.13309/) (2021)
- [Prompting Multilingual Large Language Models to Generate Code-Mixed Texts: The Case of South East Asian Languages](https://mlsanity.com/paper/2303.13592/) (2023)
- [LLM-powered Data Augmentation for Enhanced Cross-lingual Performance](https://mlsanity.com/paper/2305.14288/) (2023)
- [ASCEND: A Spontaneous Chinese-English Dataset for Code-switching in Multi-turn Conversation](https://mlsanity.com/paper/2112.06223/) (2021)
- [The Decades Progress on Code-Switching Research in NLP: A Systematic Survey on Trends and Challenges](https://mlsanity.com/paper/2212.09660/) (2022)
- [From Machine Translation to Code-Switching: Generating High-Quality Code-Switched Text](https://mlsanity.com/paper/2107.06483/) (2021)
- [HinGE: A Dataset for Generation and Evaluation of Code-Mixed Hinglish Text](https://mlsanity.com/paper/2107.03760/) (2021)
- [Deep Transfer Learning for Automatic Speech Recognition: Towards Better Generalization](https://mlsanity.com/paper/2304.14535/) (2023)
- [Code-Mixing on Sesame Street: Dawn of the Adversarial Polyglots](https://mlsanity.com/paper/2103.09593/) (2021)
- [Can You Traducir This? Machine Translation for Code-Switched Input](https://mlsanity.com/paper/2105.04846/) (2021)
- [Exploring Text-to-Text Transformers for English to Hinglish Machine Translation with Synthetic Code-Mixing](https://mlsanity.com/paper/2105.08807/) (2021)
- [Representativeness as a Forgotten Lesson for Multilingual and Code-switched Data Collection and Preparation](https://mlsanity.com/paper/2310.20470/) (2023)
- [MetaMetrics: Calibrating Metrics For Generation Tasks Using Human Preferences](https://mlsanity.com/paper/2410.02381/) (2024)
- [Enhancing Multilingual Language Model with Massive Multilingual Knowledge Triples](https://mlsanity.com/paper/2111.10962/) (2021)
- [Investigating Code-Mixed Modern Standard Arabic-Egyptian to English Machine Translation](https://mlsanity.com/paper/2105.13573/) (2021)
- [Textual Data Augmentation for Arabic-English Code-Switching Speech Recognition](https://mlsanity.com/paper/2201.02550/) (2022)
- [Lost in the Mix: Evaluating LLM Understanding of Code-Switched Text](https://mlsanity.com/paper/2506.14012/) (2025)
- [CodeMixBench: Evaluating Code-Mixing Capabilities of LLMs Across 18 Languages](https://mlsanity.com/paper/2507.18791/) (2025)
- [Generating Synthetic Text Data to Evaluate Causal Inference Methods](https://mlsanity.com/paper/2102.05638/) (2021)
- [Investigating Lexical Replacements for Arabic-English Code-Switched Data Augmentation](https://mlsanity.com/paper/2205.12649/) (2022)
- [El Volumen Louder Por Favor: Code-switching in Task-oriented Semantic Parsing](https://mlsanity.com/paper/2101.10524/) (2021)
- [Unified model for code-switching speech recognition and language identification based on a concatenated tokenizer](https://mlsanity.com/paper/2306.08753/) (2023)
- [From Human Judgements to Predictive Models: Unravelling Acceptability in Code-Mixed Sentences](https://mlsanity.com/paper/2405.05572/) (2024)
- [Exploring Methods for Building Dialects-Mandarin Code-Mixing Corpora: A Case Study in Taiwanese Hokkien](https://mlsanity.com/paper/2301.08937/) (2023)
- [Code-Switched Text Synthesis in Unseen Language Pairs](https://mlsanity.com/paper/2305.16724/) (2023)
- [Code-Mixed Probes Show How Pre-Trained Models Generalise On Code-Switched Text](https://mlsanity.com/paper/2403.04872/) (2024)
- [Speaker Information Can Guide Models to Better Inductive Biases: A Case Study On Predicting Code-Switching](https://mlsanity.com/paper/2203.08979/) (2022)
- [Linguistics Theory Meets LLM: Code-Switched Text Generation via Equivalence Constrained Large Language Models](https://mlsanity.com/paper/2410.22660/) (2024)

## Abstract (arXiv)

Training code-switched language models is difficult due to lack of data and complexity in the grammatical structure. Linguistic constraint theories have been used for decades to generate artificial code-switching sentences to cope with this issue. However, this require external word alignments or constituency parsers that create erroneous results on distant languages. We propose a sequence-to-sequence model using a copy mechanism to generate code-switching data by leveraging parallel monolingual translations from a limited source of code-switching data. The model learns how to combine words from parallel sentences and identifies when to switch one language to the other. Moreover, it captures code-switching constraints by attending and aligning the words in inputs, without requiring any external knowledge. Based on experimental results, the language model trained with the generated sentences achieves state-of-the-art performance and improves end-to-end automatic speech recognition.