# Code-switched Language Models Using Dual RNNs and Same-Source Pretraining

arXiv: https://arxiv.org/abs/1809.01962 · HTML: https://mlsanity.com/paper/1809.01962/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Sep 2018.

## Built on by (most cited first, 16 of 16)

- [The Decades Progress on Code-Switching Research in NLP: A Systematic Survey on Trends and Challenges](https://mlsanity.com/paper/2212.09660/) (2022)
- [From Machine Translation to Code-Switching: Generating High-Quality Code-Switched Text](https://mlsanity.com/paper/2107.06483/) (2021)
- [Can You Traducir This? Machine Translation for Code-Switched Input](https://mlsanity.com/paper/2105.04846/) (2021)
- [Exploring Text-to-Text Transformers for English to Hinglish Machine Translation with Synthetic Code-Mixing](https://mlsanity.com/paper/2105.08807/) (2021)
- [Investigating Code-Mixed Modern Standard Arabic-Egyptian to English Machine Translation](https://mlsanity.com/paper/2105.13573/) (2021)
- [CALCS 2021 Shared Task: Machine Translation for Code-Switched Data](https://mlsanity.com/paper/2202.09625/) (2022)
- [Contextual Label Projection for Cross-Lingual Structured Prediction](https://mlsanity.com/paper/2309.08943/) (2023)
- [Code-Switched Text Synthesis in Unseen Language Pairs](https://mlsanity.com/paper/2305.16724/) (2023)
- [SPEED++: A Multilingual Event Extraction Framework for Epidemic Prediction and Preparedness](https://mlsanity.com/paper/2410.18393/) (2024)
- [Persona-aware Generative Model for Code-mixed Language](https://mlsanity.com/paper/2309.02915/) (2023)
- [Conditioning LLMs to Generate Code-Switched Text](https://mlsanity.com/paper/2502.12924/) (2025)
- [Low-resource Machine Translation for Code-switched Kazakh-Russian Language Pair](https://mlsanity.com/paper/2503.20007/) (2025)
- [SIT at MixMT 2022: Fluent Translation Built on Giant Pre-trained Models](https://mlsanity.com/paper/2210.11670/) (2022)
- [The Effect of Alignment Objectives on Code-Switching Translation](https://mlsanity.com/paper/2309.05044/) (2023)
- [Building English ASR model with regional language support](https://mlsanity.com/paper/2503.07522/) (2025)
- [NLP Privacy Risk Identification in Social Media (NLP-PRISM): A Survey](https://mlsanity.com/paper/2602.15866/) (2026)

## Abstract (arXiv)

This work focuses on building language models (LMs) for code-switched text. We propose two techniques that significantly improve these LMs: 1) A novel recurrent neural network unit with dual components that focus on each language in the code-switched text separately 2) Pretraining the LM using synthetic text from a generative model estimated using the training data. We demonstrate the effectiveness of our proposed techniques by reporting perplexities on a Mandarin-English task and derive significant reductions in perplexity.