# Multilingual context-based pronunciation learning for Text-to-Speech

arXiv: https://arxiv.org/abs/2307.16709 · HTML: https://mlsanity.com/paper/2307.16709/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 34 references that are in the map.
- First posted on arXiv 31 Jul 2023.

## Builds on (most cited first, 15 of 15)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [ByT5 model for massively multilingual grapheme-to-phoneme conversion](https://mlsanity.com/paper/2204.03067/) (2022)
- [Applying the Transformer to Character-level Transduction](https://mlsanity.com/paper/2005.10213/) (2020)
- [Transformer based Grapheme-to-Phoneme Conversion](https://mlsanity.com/paper/2004.06338/) (2020)
- [Arabic Text Diacritization Using Deep Neural Networks](https://mlsanity.com/paper/1905.01965/) (2019)
- [Unified Mandarin TTS Front-end Based on Distilled BERT Model](https://mlsanity.com/paper/2012.15404/) (2020)
- [Massively Multilingual Neural Grapheme-to-Phoneme Conversion](https://mlsanity.com/paper/1708.01464/) (2017)
- [SoundChoice: Grapheme-to-Phoneme Models with Semantic Disambiguation](https://mlsanity.com/paper/2207.13703/) (2022)
- [A unified sequence-to-sequence front-end model for Mandarin text-to-speech synthesis](https://mlsanity.com/paper/1911.04111/) (2019)
- [Scalable Multilingual Frontend for TTS](https://mlsanity.com/paper/2004.04934/) (2020)
- [g2pW: A Conditional Weighted Softmax BERT for Polyphone Disambiguation in Mandarin](https://mlsanity.com/paper/2203.10430/) (2022)
- [Improving Arabic Diacritization by Learning to Diacritize and Translate](https://mlsanity.com/paper/2109.14150/) (2021)
- [Neural Grapheme-to-Phoneme Conversion with Pre-trained Grapheme Models](https://mlsanity.com/paper/2201.10716/) (2022)
- [A Polyphone BERT for Polyphone Disambiguation in Mandarin Chinese](https://mlsanity.com/paper/2207.12089/) (2022)

## Built on by (most cited first, 2 of 2)

- [Acquiring Pronunciation Knowledge from Transcribed Speech Audio via Multi-task Learning](https://mlsanity.com/paper/2409.09891/) (2024)
- [A Two-Step Approach for Data-Efficient French Pronunciation Learning](https://mlsanity.com/paper/2410.05698/) (2024)

## Abstract (arXiv)

Phonetic information and linguistic knowledge are an essential component of a Text-to-speech (TTS) front-end. Given a language, a lexicon can be collected offline and Grapheme-to-Phoneme (G2P) relationships are usually modeled in order to predict the pronunciation for out-of-vocabulary (OOV) words. Additionally, post-lexical phonology, often defined in the form of rule-based systems, is used to correct pronunciation within or between words. In this work we showcase a multilingual unified front-end system that addresses any pronunciation related task, typically handled by separate modules. We evaluate the proposed model on G2P conversion and other language-specific challenges, such as homograph and polyphones disambiguation, post-lexical rules and implicit diacritization. We find that the multilingual model is competitive across languages and tasks, however, some trade-offs exists when compared to equivalent monolingual solutions.