# Language-agnostic Code-Switching in Sequence-To-Sequence Speech Recognition

arXiv: https://arxiv.org/abs/2210.08992 · HTML: https://mlsanity.com/paper/2210.08992/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 55 references that are in the map.
- First posted on arXiv 17 Oct 2022.

## Builds on (most cited first, 17 of 17)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Common Voice: A Massively-Multilingual Speech Corpus](https://mlsanity.com/paper/1912.06670/) (2019)
- [Attention-Based Models for Speech Recognition](https://mlsanity.com/paper/1506.07503/) (2015)
- [How2: A Large-scale Dataset for Multimodal Language Understanding](https://mlsanity.com/paper/1811.00347/) (2018)
- [Multilingual and code-switching ASR challenges for low resource Indian languages](https://mlsanity.com/paper/2104.00235/) (2021)
- [On the End-to-End Solution to Mandarin-English Code-switching Speech Recognition](https://mlsanity.com/paper/1811.00241/) (2018)
- [Improving sequence-to-sequence speech recognition training with on-the-fly data augmentation](https://mlsanity.com/paper/1910.13296/) (2019)
- [Code-switching Sentence Generation by Generative Adversarial Networks and its Application to Data Augmentation](https://mlsanity.com/paper/1811.02356/) (2018)
- [The MGB-2 Challenge: Arabic Multi-Dialect Broadcast Media Recognition](https://mlsanity.com/paper/1609.05625/) (2016)
- [Data Augmentation for End-to-end Code-switching Speech Recognition](https://mlsanity.com/paper/2011.02160/) (2020)
- [Towards End-to-End Code-Switching Speech Recognition](https://mlsanity.com/paper/1810.13091/) (2018)
- [Investigations on Speech Recognition Systems for Low-Resource Dialectal Arabic-English Code-Switching Speech](https://mlsanity.com/paper/2108.12881/) (2021)
- [Acoustic and Textual Data Augmentation for Improved ASR of Code-Switching Speech](https://mlsanity.com/paper/1807.10945/) (2018)
- [Learning to Recognize Code-switched Speech Without Forgetting Monolingual Speech Recognition](https://mlsanity.com/paper/2006.00782/) (2020)
- [Study of Semi-supervised Approaches to Improving English-Mandarin Code-Switching Speech Recognition](https://mlsanity.com/paper/1806.06200/) (2018)
- [A Study of Multilingual End-to-End Speech Recognition for Kazakh, Russian, and English](https://mlsanity.com/paper/2108.01280/) (2021)

## Built on by (most cited first, 7 of 7)

- [PIER: A Novel Metric for Evaluating What Matters in Code-Switching](https://mlsanity.com/paper/2501.09512/) (2025)
- [Weight Factorization and Centralization for Continual Learning in Speech Recognition](https://mlsanity.com/paper/2506.16574/) (2025)
- [A Survey of Code-switched Arabic NLP: Progress, Challenges, and Future Directions](https://mlsanity.com/paper/2501.13419/) (2025)
- [Adapting Language Balance in Code-Switching Speech](https://mlsanity.com/paper/2510.18724/) (2025)
- [Bayesian Low-Rank Factorization for Robust Model Adaptation](https://mlsanity.com/paper/2510.18723/) (2025)
- [MUSCAT: MUltilingual, SCientific ConversATion Benchmark](https://mlsanity.com/paper/2604.15929/) (2026)
- [Adding Robust Code-Switching Capabilities to High Performance Multilingual ASR](https://mlsanity.com/paper/2606.21990/) (2026)

## Abstract (arXiv)

Code-Switching (CS) is referred to the phenomenon of alternately using words and phrases from different languages. While today's neural end-to-end (E2E) models deliver state-of-the-art performances on the task of automatic speech recognition (ASR) it is commonly known that these systems are very data-intensive. However, there is only a few transcribed and aligned CS speech available. To overcome this problem and train multilingual systems which can transcribe CS speech, we propose a simple yet effective data augmentation in which audio and corresponding labels of different source languages are concatenated. By using this training data, our E2E model improves on transcribing CS speech. It also surpasses monolingual models on monolingual tests. The results show that this augmentation technique can even improve the model's performance on inter-sentential language switches not seen during training by 5,03% WER.