# AsyncSwitch: Asynchronous Text-Speech Adaptation for Code-Switched ASR

arXiv: https://arxiv.org/abs/2506.14190 · HTML: https://mlsanity.com/paper/2506.14190/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 36 references that are in the map.
- First posted on arXiv 17 Jun 2025.

## Builds on (most cited first, 18 of 18)

- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition](https://mlsanity.com/paper/1904.08779/) (2019)
- [MUSAN: A Music, Speech, and Noise Corpus](https://mlsanity.com/paper/1510.08484/) (2015)
- [Seamless: Multilingual Expressive and Streaming Speech Translation](https://mlsanity.com/paper/2312.05187/) (2023)
- [Internal Language Model Estimation for Domain-Adaptive End-to-End Speech Recognition](https://mlsanity.com/paper/2011.01991/) (2020)
- [A Survey on Speech Large Language Models for Understanding](https://mlsanity.com/paper/2410.18908/) (2024)
- [Careless Whisper: Speech-to-Text Hallucination Harms](https://mlsanity.com/paper/2402.08021/) (2024)
- [SpeechLM: Enhanced Speech Pre-Training with Unpaired Textual Data](https://mlsanity.com/paper/2209.15329/) (2022)
- [Meta-Transfer Learning for Code-Switched Speech Recognition](https://mlsanity.com/paper/2004.14228/) (2020)
- [Data Augmentation for End-to-end Code-switching Speech Recognition](https://mlsanity.com/paper/2011.02160/) (2020)
- [Improving Low Resource Code-switched ASR using Augmented Code-switched TTS](https://mlsanity.com/paper/2010.05549/) (2020)
- [Building a Unified Code-Switching ASR System for South African Languages](https://mlsanity.com/paper/1807.10949/) (2018)
- [Adapting OpenAI's Whisper for Speech Recognition on Code-Switch Mandarin-English SEAME and ASRU2019 Datasets](https://mlsanity.com/paper/2311.17382/) (2023)
- [MERaLiON-AudioLLM: Bridging Audio and Language with Large Language Models](https://mlsanity.com/paper/2412.09818/) (2024)
- [Linear Interpolation In Parameter Space is Good Enough for Fine-Tuned Language Models](https://mlsanity.com/paper/2211.12092/) (2022)
- [Text-only Domain Adaptation using Unified Speech-Text Representation in Transducer](https://mlsanity.com/paper/2306.04076/) (2023)
- [Leave No Knowledge Behind During Knowledge Distillation: Towards Practical and Effective Knowledge Distillation for Code-Switching ASR Using Realistic Data](https://mlsanity.com/paper/2407.10603/) (2024)
- [Whisper-LM: Improving ASR Models with Language Models for Low-Resource Languages](https://mlsanity.com/paper/2503.23542/) (2025)

## Built on by (most cited first, 2 of 2)

- [Towards Truly Multilingual ASR: Generalizing Code-Switching ASR to Unseen Language Pairs](https://mlsanity.com/paper/2606.05846/) (2026)
- [Code-Switching Detection via Language-Selective Dimensions in Speech Foundation Models](https://mlsanity.com/paper/2610.18726/) (2026)

## Abstract (arXiv)

Developing code-switched ASR systems is challenging due to language ambiguity and limited exposure to multilingual, code-switched data, while collecting such speech is costly. Prior work generates synthetic audio from text, but these methods are computationally intensive and hard to scale. We introduce AsyncSwitch, a novel asynchronous adaptation framework that leverages large-scale, text-rich web data to pre-expose ASR models to diverse code-switched domains before fine-tuning on paired speech-text corpora. Our three-stage process (1) trains decoder self-attention and feedforward layers on code-switched text, (2) aligns decoder and encoder via cross-attention using limited speech-text data, and (3) fully fine-tunes the entire model. Experiments with Whisper on Malay-English code-switching demonstrate a 9.02% relative WER reduction, while improving monolingual performance in Singlish, Malay, and other English variants.