# Pronunciation Generation for Foreign Language Words in Intra-Sentential Code-Switching Speech Recognition

arXiv: https://arxiv.org/abs/2210.14691 · HTML: https://mlsanity.com/paper/2210.14691/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 6 of its 47 references that are in the map.
- First posted on arXiv 26 Oct 2022.

## Builds on (most cited first, 6 of 6)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [AISHELL-1: An Open-Source Mandarin Speech Corpus and A Speech Recognition Baseline](https://mlsanity.com/paper/1709.05522/) (2017)
- [Study of Semi-supervised Approaches to Improving English-Mandarin Code-Switching Speech Recognition](https://mlsanity.com/paper/1806.06200/) (2018)
- [Acoustic data-driven lexicon learning based on a greedy pronunciation selection framework](https://mlsanity.com/paper/1706.03747/) (2017)
- [An enhanced Tree-LSTM architecture for sentence semantic modeling using typed dependencies](https://mlsanity.com/paper/2002.07775/) (2020)
- [Exploring Retraining-Free Speech Recognition for Intra-sentential Code-Switching](https://mlsanity.com/paper/2109.00921/) (2021)

## Abstract (arXiv)

Code-Switching refers to the phenomenon of switching languages within a sentence or discourse. However, limited code-switching , different language phoneme-sets and high rebuilding costs throw a challenge to make the specialized acoustic model for code-switching speech recognition. In this paper, we make use of limited code-switching data as driving materials and explore a shortcut to quickly develop intra-sentential code-switching recognition skill on the commissioned native language acoustic model, where we propose a data-driven method to make the seed lexicon which is used to train grapheme-to-phoneme model to predict mapping pronunciations for foreign language word in code-switching sentences. The core work of the data-driven technology in this paper consists of a phonetic decoding method and different selection methods. And for imbalanced word-level driving materials problem, we have an internal assistance inspiration that learning the good pronunciation rules in the words that possess sufficient materials using the grapheme-to-phoneme model to help the scarce. Our experiments show that the Mixed Error Rate in intra-sentential Chinese-English code-switching recognition reduced from 29.15%, acquired on the pure Chinese recognizer, to 12.13% by adding foreign language words' pronunciation through our data-driven approach, and finally get the best result 11.14% with the combination of different selection methods and internal assistance tactic.