# TSPC: A Two-Stage Phoneme-Centric Architecture for code-switching Vietnamese-English Speech Recognition

arXiv: https://arxiv.org/abs/2509.05983 · HTML: https://mlsanity.com/paper/2509.05983/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 26 references that are in the map.
- First posted on arXiv 7 Sep 2025.

## Builds on (most cited first, 13 of 13)

- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [Common Voice: A Massively-Multilingual Speech Corpus](https://mlsanity.com/paper/1912.06670/) (2019)
- [Scaling Speech Technology to 1,000+ Languages](https://mlsanity.com/paper/2305.13516/) (2023)
- [Deep context: end-to-end contextual speech recognition](https://mlsanity.com/paper/1808.02480/) (2018)
- [Neural Machine Translation: A Review and Survey](https://mlsanity.com/paper/1912.02047/) (2019)
- [On the End-to-End Solution to Mandarin-English Code-switching Speech Recognition](https://mlsanity.com/paper/1811.00241/) (2018)
- [PhoWhisper: Automatic Speech Recognition for Vietnamese](https://mlsanity.com/paper/2406.02555/) (2024)
- [Enhancing Code-Switching Speech Recognition with LID-Based Collaborative Mixture of Experts Model](https://mlsanity.com/paper/2409.02050/) (2024)
- [AdaCS: Adaptive Normalization for Enhanced Code-Switching ASR](https://mlsanity.com/paper/2501.07102/) (2025)
- [A Multi-Encoder Frozen-Decoder Approach for Fine-Tuning Large Language Models](https://mlsanity.com/paper/2501.07818/) (2025)
- [Whisper based Cross-Lingual Phoneme Recognition between Vietnamese and English](https://mlsanity.com/paper/2508.19270/) (2025)

## Abstract (arXiv)

Code-switching (CS) presents a significant challenge for general Auto-Speech Recognition (ASR) systems. Existing methods often fail to capture the sub tle phonological shifts inherent in CS scenarios. The challenge is particu larly difficult for language pairs like Vietnamese and English, where both distinct phonological features and the ambiguity arising from similar sound recognition are present. In this paper, we propose a novel architecture for Vietnamese-English CS ASR, a Two-Stage Phoneme-Centric model (TSPC). TSPC adopts a phoneme-centric approach based on an extended Vietnamese phoneme set as an intermediate representation for mixed-lingual modeling, while remaining efficient under low computational-resource constraints. Ex perimental results demonstrate that TSPC consistently outperforms exist ing baselines, including PhoWhisper-base, in Vietnamese-English CS ASR, achieving a significantly lower word error rate of 19.06% with reduced train ing resources. Furthermore, the phonetic-based two-stage architecture en ables phoneme adaptation and language conversion to enhance ASR perfor mance in complex CS Vietnamese-English ASR scenarios.