# Neural Lexicon Reader: Reduce Pronunciation Errors in End-to-end TTS by Leveraging External Textual Knowledge

arXiv: https://arxiv.org/abs/2110.09698 · HTML: https://mlsanity.com/paper/2110.09698/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 30 references that are in the map.
- First posted on arXiv 19 Oct 2021.

## Builds on (most cited first, 14 of 14)

- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [Language Models as Knowledge Bases?](https://mlsanity.com/paper/1909.01066/) (2019)
- [Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictions](https://mlsanity.com/paper/1712.05884/) (2017)
- [Reading Wikipedia to Answer Open-Domain Questions](https://mlsanity.com/paper/1704.00051/) (2017)
- [Bidirectional Attention Flow for Machine Comprehension](https://mlsanity.com/paper/1611.01603/) (2016)
- [GlossBERT: BERT for Word Sense Disambiguation with Gloss Knowledge](https://mlsanity.com/paper/1908.07245/) (2019)
- [PnG BERT: Augmented BERT on Phonemes and Graphemes for Neural TTS](https://mlsanity.com/paper/2103.15060/) (2021)
- [Commonsense for Generative Multi-Hop Question Answering Tasks](https://mlsanity.com/paper/1809.06309/) (2018)
- [Lexicon Enhanced Chinese Sequence Labeling Using BERT Adapter](https://mlsanity.com/paper/2105.07148/) (2021)
- [Multilingual Byte2Speech Models for Scalable Low-resource Speech Synthesis](https://mlsanity.com/paper/2103.03541/) (2021)
- [Memory-augmented Neural Machine Translation](https://mlsanity.com/paper/1708.02005/) (2017)
- [Homograph Disambiguation Through Selective Diacritic Restoration](https://mlsanity.com/paper/1912.04479/) (2019)
- [Investigation of learning abilities on linguistic features in sequence-to-sequence text-to-speech synthesis](https://mlsanity.com/paper/2005.10390/) (2020)
- [Polyphone Disambiguation in Mandarin Chinese with Semi-Supervised Learning](https://mlsanity.com/paper/2102.00621/) (2021)

## Built on by (most cited first, 3 of 3)

- [Dict-TTS: Learning to Pronounce with Prior Dictionary Knowledge for Text-to-Speech](https://mlsanity.com/paper/2206.02147/) (2022)
- [GRAFT: Grafted Reference Audio for Fine-grained Pronunciation in Zero-shot Text-to-Speech](https://mlsanity.com/paper/2607.02633/) (2026)
- [A survey of AI-generated voices and their detection](https://mlsanity.com/paper/2608.15411/) (2026)

## Abstract (arXiv)

End-to-end TTS requires a large amount of speech/text paired data to cover all necessary knowledge, particularly how to pronounce different words in diverse contexts, so that a neural model may learn such knowledge accordingly. But in real applications, such high demand of training data is hard to be satisfied and additional knowledge often needs to be injected manually. For example, to capture pronunciation knowledge on languages without regular orthography, a complicated grapheme-to-phoneme pipeline needs to be built based on a large structured pronunciation lexicon, leading to extra, sometimes high, costs to extend neural TTS to such languages. In this paper, we propose a framework to learn to automatically extract knowledge from unstructured external resources using a novel Token2Knowledge attention module. The framework is applied to build a TTS model named Neural Lexicon Reader that extracts pronunciations from raw lexicon texts in an end-to-end manner. Experiments show the proposed model significantly reduces pronunciation errors in low-resource, end-to-end Chinese TTS, and the lexicon-reading capability can be transferred to other languages with a smaller amount of data.