# Transformer based Grapheme-to-Phoneme Conversion

arXiv: https://arxiv.org/abs/2004.06338 · HTML: https://mlsanity.com/paper/2004.06338/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Apr 2020.

## Built on by (most cited first, 30 of 32)

- [ByT5 model for massively multilingual grapheme-to-phoneme conversion](https://mlsanity.com/paper/2204.03067/) (2022)
- [SoundChoice: Grapheme-to-Phoneme Models with Semantic Disambiguation](https://mlsanity.com/paper/2207.13703/) (2022)
- [Phrase break prediction with bidirectional encoder representations in Japanese text-to-speech synthesis](https://mlsanity.com/paper/2104.12395/) (2021)
- [LLM-Powered Grapheme-to-Phoneme Conversion: Benchmark and Case Study](https://mlsanity.com/paper/2409.08554/) (2024)
- [r-G2P: Evaluating and Enhancing Robustness of Grapheme to Phoneme Conversion by Controlled noise introducing and Contextual information incorporation](https://mlsanity.com/paper/2202.11194/) (2022)
- [PhonologyBench: Evaluating Phonological Skills of Large Language Models](https://mlsanity.com/paper/2404.02456/) (2024)
- [On Biasing Transformer Attention Towards Monotonicity](https://mlsanity.com/paper/2104.03945/) (2021)
- [LASER: An LLM-based ASR Scoring and Evaluation Rubric](https://mlsanity.com/paper/2510.07437/) (2025)
- [Neural Grapheme-to-Phoneme Conversion with Pre-trained Grapheme Models](https://mlsanity.com/paper/2201.10716/) (2022)
- [A unified front-end framework for English text-to-speech synthesis](https://mlsanity.com/paper/2305.10666/) (2023)
- [Connecting the Persian-speaking World through Transliteration](https://mlsanity.com/paper/2502.20047/) (2025)
- [ParsTranslit: Truly Versatile Tajik-Farsi Transliteration](https://mlsanity.com/paper/2510.07520/) (2025)
- [Grapheme-to-Phoneme Transformer Model for Transfer Learning Dialects](https://mlsanity.com/paper/2104.04091/) (2021)
- [Evaluating Speech Synthesis by Training Recognizers on Synthetic Speech](https://mlsanity.com/paper/2310.00706/) (2023)
- [Data-driven grapheme-to-phoneme representations for a lexicon-free text-to-speech](https://mlsanity.com/paper/2401.10465/) (2024)
- [Searching for Search Errors in Neural Morphological Inflection](https://mlsanity.com/paper/2102.08424/) (2021)
- [A Novel Chinese Dialect TTS Frontend with Non-Autoregressive Neural Machine Translation](https://mlsanity.com/paper/2206.04922/) (2022)
- [LiteG2P: A fast, light and high accuracy model for grapheme-to-phoneme conversion](https://mlsanity.com/paper/2303.01086/) (2023)
- [Improving grapheme-to-phoneme conversion by learning pronunciations from speech recordings](https://mlsanity.com/paper/2307.16643/) (2023)
- [Multilingual context-based pronunciation learning for Text-to-Speech](https://mlsanity.com/paper/2307.16709/) (2023)
- [Improving Grapheme-to-Phoneme Conversion through In-Context Knowledge Retrieval with Large Language Models](https://mlsanity.com/paper/2411.07563/) (2024)
- [BanglaIPA: Towards Robust Text-to-IPA Transcription with Contextual Rewriting in Bengali](https://mlsanity.com/paper/2601.01778/) (2026)
- [Multi-Module G2P Converter for Persian Focusing on Relations between Words](https://mlsanity.com/paper/2208.01371/) (2022)
- [Mitigating the Exposure Bias in Sentence-Level Grapheme-to-Phoneme (G2P) Transduction](https://mlsanity.com/paper/2308.08442/) (2023)
- [Scaling Human and G2P Supervision for Robust Phonetic Transcription](https://mlsanity.com/paper/2606.16019/) (2026)
- [Towards Stress-Aware Sentence-Level Filipino G2P With Weakly-Supervised ByT5 Fine-Tuning](https://mlsanity.com/paper/2609.09974/) (2026)
- [Neural String Edit Distance](https://mlsanity.com/paper/2104.08388/) (2021)
- [Learning to pronounce as measuring cross-lingual joint orthography-phonology complexity](https://mlsanity.com/paper/2202.00794/) (2022)
- [A Two-Step Approach for Data-Efficient French Pronunciation Learning](https://mlsanity.com/paper/2410.05698/) (2024)
- [GraphemeAug: A Systematic Approach to Synthesized Hard Negative Keyword Spotting Examples](https://mlsanity.com/paper/2505.14814/) (2025)

## Abstract (arXiv)

Attention mechanism is one of the most successful techniques in deep learning based Natural Language Processing (NLP). The transformer network architecture is completely based on attention mechanisms, and it outperforms sequence-to-sequence models in neural machine translation without recurrent and convolutional layers. Grapheme-to-phoneme (G2P) conversion is a task of converting letters (grapheme sequence) to their pronunciations (phoneme sequence). It plays a significant role in text-to-speech (TTS) and automatic speech recognition (ASR) systems. In this paper, we investigate the application of transformer architecture to G2P conversion and compare its performance with recurrent and convolutional neural network based approaches. Phoneme and word error rates are evaluated on the CMUDict dataset for US English and the NetTalk dataset. The results show that transformer based G2P outperforms the convolutional-based approach in terms of word error rate and our results significantly exceeded previous recurrent approaches (without attention) regarding word and phoneme error rates on both datasets. Furthermore, the size of the proposed model is much smaller than the size of the previous approaches.