# Improving grapheme-to-phoneme conversion by learning pronunciations from speech recordings

arXiv: https://arxiv.org/abs/2307.16643 · HTML: https://mlsanity.com/paper/2307.16643/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 37 references that are in the map.
- First posted on arXiv 31 Jul 2023.

## Builds on (most cited first, 16 of 16)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [mT5: A massively multilingual pre-trained text-to-text transformer](https://mlsanity.com/paper/2010.11934/) (2020)
- [Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictions](https://mlsanity.com/paper/1712.05884/) (2017)
- [OpenNMT: Open-source Toolkit for Neural Machine Translation](https://mlsanity.com/paper/1709.03815/) (2017)
- [Universal Phone Recognition with a Multilingual Allophone System](https://mlsanity.com/paper/2002.11800/) (2020)
- [ByT5 model for massively multilingual grapheme-to-phoneme conversion](https://mlsanity.com/paper/2204.03067/) (2022)
- [Transformer based Grapheme-to-Phoneme Conversion](https://mlsanity.com/paper/2004.06338/) (2020)
- [Pomegranate: fast and flexible probabilistic modeling in python](https://mlsanity.com/paper/1711.00137/) (2017)
- [WavThruVec: Latent speech representation as intermediate features for neural speech synthesis](https://mlsanity.com/paper/2203.16930/) (2022)
- [Massively Multilingual Neural Grapheme-to-Phoneme Conversion](https://mlsanity.com/paper/1708.01464/) (2017)
- [Deciphering Speech: a Zero-Resource Approach to Cross-Lingual Transfer in ASR](https://mlsanity.com/paper/2111.06799/) (2021)
- [Jointly Learning to Align and Convert Graphemes to Phonemes with Neural Attention Models](https://mlsanity.com/paper/1610.06540/) (2016)
- [Discovering Phonetic Inventories with Crosslingual Automatic Speech Recognition](https://mlsanity.com/paper/2201.11207/) (2022)
- [Neural Machine Translation for Multilingual Grapheme-to-Phoneme Conversion](https://mlsanity.com/paper/2006.14194/) (2020)
- [Neural Grapheme-to-Phoneme Conversion with Pre-trained Grapheme Models](https://mlsanity.com/paper/2201.10716/) (2022)
- [Grapheme-to-Phoneme Transformer Model for Transfer Learning Dialects](https://mlsanity.com/paper/2104.04091/) (2021)

## Built on by (most cited first, 2 of 2)

- [Acquiring Pronunciation Knowledge from Transcribed Speech Audio via Multi-task Learning](https://mlsanity.com/paper/2409.09891/) (2024)
- [ReNikud: Audio-Supervised Hebrew Grapheme-to-Phoneme Conversion](https://mlsanity.com/paper/2606.20179/) (2026)

## Abstract (arXiv)

The Grapheme-to-Phoneme (G2P) task aims to convert orthographic input into a discrete phonetic representation. G2P conversion is beneficial to various speech processing applications, such as text-to-speech and speech recognition. However, these tend to rely on manually-annotated pronunciation dictionaries, which are often time-consuming and costly to acquire. In this paper, we propose a method to improve the G2P conversion task by learning pronunciation examples from audio recordings. Our approach bootstraps a G2P with a small set of annotated examples. The G2P model is used to train a multilingual phone recognition system, which then decodes speech recordings with a phonetic representation. Given hypothesized phoneme labels, we learn pronunciation dictionaries for out-of-vocabulary words, and we use those to re-train the G2P system. Results indicate that our approach consistently improves the phone error rate of G2P systems across languages and amount of available data.