# Code-Switched Text Synthesis in Unseen Language Pairs

arXiv: https://arxiv.org/abs/2305.16724 · HTML: https://mlsanity.com/paper/2305.16724/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 49 references that are in the map.
- First posted on arXiv 26 May 2023.

## Builds on (most cited first, 26 of 26)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Parameter-Efficient Transfer Learning for NLP](https://mlsanity.com/paper/1902.00751/) (2019)
- [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://mlsanity.com/paper/2101.00190/) (2021)
- [Cross-lingual Language Model Pretraining](https://mlsanity.com/paper/1901.07291/) (2019)
- [Towards a Unified View of Parameter-Efficient Transfer Learning](https://mlsanity.com/paper/2110.04366/) (2021)
- [Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation](https://mlsanity.com/paper/1611.04558/) (2016)
- [AdapterHub: A Framework for Adapting Transformers](https://mlsanity.com/paper/2007.07779/) (2020)
- [Multilingual Translation with Extensible Multilingual Pretraining and Finetuning](https://mlsanity.com/paper/2008.00401/) (2020)
- [Word Alignment by Fine-tuning Embeddings on Parallel Corpora](https://mlsanity.com/paper/2101.08231/) (2021)
- [MASSIVE: A 1M-Example Multilingual Natural Language Understanding Dataset with 51 Typologically-Diverse Languages](https://mlsanity.com/paper/2204.08582/) (2022)
- [GLUECoS : An Evaluation Benchmark for Code-Switched NLP](https://mlsanity.com/paper/2004.12376/) (2020)
- [Toward Multilingual Neural Machine Translation with Universal Encoder and Decoder](https://mlsanity.com/paper/1611.04798/) (2016)
- [A Survey of Code-switched Speech and Language Processing](https://mlsanity.com/paper/1904.00784/) (2019)
- [Code-Switched Language Models Using Neural Based Synthetic Data from Parallel Sentences](https://mlsanity.com/paper/1909.08582/) (2019)
- [Multilingual Generative Language Models for Zero-Shot Cross-Lingual Event Argument Extraction](https://mlsanity.com/paper/2203.08308/) (2022)
- [The Decades Progress on Code-Switching Research in NLP: A Systematic Survey on Trends and Challenges](https://mlsanity.com/paper/2212.09660/) (2022)
- [From Machine Translation to Code-Switching: Generating High-Quality Code-Switched Text](https://mlsanity.com/paper/2107.06483/) (2021)
- [Code-switching Sentence Generation by Generative Adversarial Networks and its Application to Data Augmentation](https://mlsanity.com/paper/1811.02356/) (2018)
- [Improving Zero-Shot Cross-Lingual Transfer Learning via Robust Training](https://mlsanity.com/paper/2104.08645/) (2021)
- [Self-supervised Learning for Pre-Training 3D Point Clouds: A Survey](https://mlsanity.com/paper/2305.04691/) (2023)
- [Code-switched Language Models Using Dual RNNs and Same-Source Pretraining](https://mlsanity.com/paper/1809.01962/) (2018)
- [Multilingual Machine Translation with Hyper-Adapters](https://mlsanity.com/paper/2205.10835/) (2022)
- [Language Modeling for Code-Switching: Evaluation, Integration of Monolingual Data, and Discriminative Training](https://mlsanity.com/paper/1810.11895/) (2018)
- [A Deep Generative Model for Code-Switched Text](https://mlsanity.com/paper/1906.08972/) (2019)
- [Improving Systematic Generalization Through Modularity and Augmentation](https://mlsanity.com/paper/2202.10745/) (2022)
- [Grammatical Constraints on Intra-sentential Code-Switching: From Theories to Working Models](https://mlsanity.com/paper/1612.04538/) (2016)

## Built on by (most cited first, 7 of 7)

- [CodeMixBench: Evaluating Code-Mixing Capabilities of LLMs Across 18 Languages](https://mlsanity.com/paper/2507.18791/) (2025)
- [Contextual Label Projection for Cross-Lingual Structured Prediction](https://mlsanity.com/paper/2309.08943/) (2023)
- [Code-Mixer Ya Nahi: Novel Approaches to Measuring Multilingual LLMs' Code-Mixing Capabilities](https://mlsanity.com/paper/2410.11079/) (2024)
- [Beyond Monolingual Assumptions: A Survey of Code-Switched NLP in the Era of Large Language Models across Modalities](https://mlsanity.com/paper/2510.07037/) (2025)
- [Multilingual Controlled Generation And Gold-Standard-Agnostic Evaluation of Code-Mixed Sentences](https://mlsanity.com/paper/2410.10580/) (2024)
- [Conditioning LLMs to Generate Code-Switched Text](https://mlsanity.com/paper/2502.12924/) (2025)
- [Code Mixologist : A Practitioner's Guide to Building Code-Mixed LLMs](https://mlsanity.com/paper/2602.11181/) (2026)

## Abstract (arXiv)

Existing efforts on text synthesis for code-switching mostly require training on code-switched texts in the target language pairs, limiting the deployment of the models to cases lacking code-switched data. In this work, we study the problem of synthesizing code-switched texts for language pairs absent from the training data. We introduce GLOSS, a model built on top of a pre-trained multilingual machine translation model (PMMTM) with an additional code-switching module. This module, either an adapter or extra prefixes, learns code-switching patterns from code-switched data during training, while the primary component of GLOSS, i.e., the PMMTM, is frozen. The design of only adjusting the code-switching module prevents our model from overfitting to the constrained training data for code-switching. Hence, GLOSS exhibits the ability to generalize and synthesize code-switched texts across a broader spectrum of language pairs. Additionally, we develop a self-training algorithm on target language pairs further to enhance the reliability of GLOSS. Automatic evaluations on four language pairs show that GLOSS achieves at least 55% relative BLEU and METEOR scores improvements compared to strong baselines. Human evaluations on two language pairs further validate the success of GLOSS.