# Simple yet Effective Code-Switching Language Identification with Multitask Pre-Training and Transfer Learning

arXiv: https://arxiv.org/abs/2305.19759 · HTML: https://mlsanity.com/paper/2305.19759/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 44 references that are in the map.
- First posted on arXiv 31 May 2023.

## Builds on (most cited first, 18 of 18)

- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition](https://mlsanity.com/paper/1904.08779/) (2019)
- [XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scale](https://mlsanity.com/paper/2111.09296/) (2021)
- [Unsupervised Cross-lingual Representation Learning for Speech Recognition](https://mlsanity.com/paper/2006.13979/) (2020)
- [AISHELL-1: An Open-Source Mandarin Speech Corpus and A Speech Recognition Baseline](https://mlsanity.com/paper/1709.05522/) (2017)
- [mSLAM: Massively multilingual joint pre-training for speech and text](https://mlsanity.com/paper/2202.01374/) (2022)
- [Towards One Model to Rule All: Multilingual Strategy for Dialectal Code-Switching Arabic ASR](https://mlsanity.com/paper/2105.14779/) (2021)
- [A Configurable Multilingual Model is All You Need to Recognize All Languages](https://mlsanity.com/paper/2107.05876/) (2021)
- [End-to-End Speech Translation for Code Switched Speech](https://mlsanity.com/paper/2204.05076/) (2022)
- [Gradual Fine-Tuning for Low-Resource Domain Adaptation](https://mlsanity.com/paper/2103.02205/) (2021)
- [Streaming End-to-End Multilingual Speech Recognition with Joint Language Identification](https://mlsanity.com/paper/2209.06058/) (2022)
- [Can You Traducir This? Machine Translation for Code-Switched Input](https://mlsanity.com/paper/2105.04846/) (2021)
- [Arabic Code-Switching Speech Recognition using Monolingual Data](https://mlsanity.com/paper/2107.01573/) (2021)
- [Rnn-transducer with language bias for end-to-end Mandarin-English code-switching speech recognition](https://mlsanity.com/paper/2002.08126/) (2020)
- [Textual Data Augmentation for Arabic-English Code-Switching Speech Recognition](https://mlsanity.com/paper/2201.02550/) (2022)
- [Language Identification Using Deep Convolutional Recurrent Neural Networks](https://mlsanity.com/paper/1708.04811/) (2017)
- [Language Agnostic Code-Mixing Data Augmentation by Predicting Linguistic Patterns](https://mlsanity.com/paper/2211.07628/) (2022)

## Built on by (most cited first, 2 of 2)

- [Leveraging Zipformer Model for Effective Language Identification in Code-Switched Child-Directed Speech](https://mlsanity.com/paper/2508.09430/) (2025)
- [Code-Switching Detection via Language-Selective Dimensions in Speech Foundation Models](https://mlsanity.com/paper/2610.18726/) (2026)

## Abstract (arXiv)

Code-switching, also called code-mixing, is the linguistics phenomenon where in casual settings, multilingual speakers mix words from different languages in one utterance. Due to its spontaneous nature, code-switching is extremely low-resource, which makes it a challenging problem for language and speech processing tasks. In such contexts, Code-Switching Language Identification (CSLID) becomes a difficult but necessary task if we want to maximally leverage existing monolingual tools for other tasks. In this work, we propose two novel approaches toward improving language identification accuracy on an English-Mandarin child-directed speech dataset. Our methods include a stacked Residual CNN+GRU model and a multitask pre-training approach to use Automatic Speech Recognition (ASR) as an auxiliary task for CSLID. Due to the low-resource nature of code-switching, we also employ careful silver data creation using monolingual corpora in both languages and up-sampling as data augmentation. We focus on English-Mandarin code-switched data, but our method works on any language pair. Our best model achieves a balanced accuracy of 0.781 on a real English-Mandarin code-switching child-directed speech corpus and outperforms the previous baseline by 55.3%.