# Non-native English lexicon creation for bilingual speech synthesis

arXiv: https://arxiv.org/abs/2106.10870 · HTML: https://mlsanity.com/paper/2106.10870/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 17 references that are in the map.
- First posted on arXiv 21 Jun 2021.

## Builds on (most cited first, 5 of 5)

- [WaveNet: A Generative Model for Raw Audio](https://mlsanity.com/paper/1609.03499/) (2016)
- [Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictions](https://mlsanity.com/paper/1712.05884/) (2017)
- [Parallel WaveGAN: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogram](https://mlsanity.com/paper/1910.11480/) (2019)
- [ESPnet-TTS: Unified, Reproducible, and Integratable Open Source End-to-End Text-to-Speech Toolkit](https://mlsanity.com/paper/1910.10909/) (2019)
- [Learning pronunciation from a foreign language in speech synthesis networks](https://mlsanity.com/paper/1811.09364/) (2018)

## Built on by (most cited first, 3 of 3)

- [Improve Bilingual TTS Using Dynamic Language and Phonology Embedding](https://mlsanity.com/paper/2212.03435/) (2022)
- [Context-based out-of-vocabulary word recovery for ASR systems in Indian languages](https://mlsanity.com/paper/2206.04305/) (2022)
- [An Investigation of Indian Native Language Phonemic Influences on L2 English Pronunciations](https://mlsanity.com/paper/2212.09284/) (2022)

## Abstract (arXiv)

Bilingual English speakers speak English as one of their languages. Their English is of a non-native kind, and their conversations are of a code-mixed fashion. The intelligibility of a bilingual text-to-speech (TTS) system for such non-native English speakers depends on a lexicon that captures the phoneme sequence used by non-native speakers. However, due to the lack of non-native English lexicon, existing bilingual TTS systems employ native English lexicons that are widely available, in addition to their native language lexicon. Due to the inconsistency between the non-native English pronunciation in the audio and native English lexicon in the text, the intelligibility of synthesized speech in such TTS systems is significantly reduced. This paper is motivated by the knowledge that the native language of the speaker highly influences non-native English pronunciation. We propose a generic approach to obtain rules based on letter to phoneme alignment to map native English lexicon to their non-native version. The effectiveness of such mapping is studied by comparing bilingual (Indian English and Hindi) TTS systems trained with and without the proposed rules. The subjective evaluation shows that the bilingual TTS system trained with the proposed non-native English lexicon rules obtains a 6% absolute improvement in preference.