# Representation Mixing for TTS Synthesis

arXiv: https://arxiv.org/abs/1811.07240 · HTML: https://mlsanity.com/paper/1811.07240/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Nov 2018.

## Built on by (most cited first, 7 of 7)

- [A Survey on Neural Speech Synthesis](https://mlsanity.com/paper/2106.15561/) (2021)
- [PnG BERT: Augmented BERT on Phonemes and Graphemes for Neural TTS](https://mlsanity.com/paper/2103.15060/) (2021)
- [SoundChoice: Grapheme-to-Phoneme Models with Semantic Disambiguation](https://mlsanity.com/paper/2207.13703/) (2022)
- [Mixed-Phoneme BERT: Improving BERT with Mixed Phoneme and Sup-Phoneme Representations for Text to Speech](https://mlsanity.com/paper/2203.17190/) (2022)
- [Phrase break prediction with bidirectional encoder representations in Japanese text-to-speech synthesis](https://mlsanity.com/paper/2104.12395/) (2021)
- [Zero-Shot Long-Form Voice Cloning with Dynamic Convolution Attention](https://mlsanity.com/paper/2201.10375/) (2022)
- [R-MelNet: Reduced Mel-Spectral Modeling for Neural TTS](https://mlsanity.com/paper/2206.15276/) (2022)

## Abstract (arXiv)

Recent character and phoneme-based parametric TTS systems using deep learning have shown strong performance in natural speech generation. However, the choice between character or phoneme input can create serious limitations for practical deployment, as direct control of pronunciation is crucial in certain cases. We demonstrate a simple method for combining multiple types of linguistic information in a single encoder, named representation mixing, enabling flexible choice between character, phoneme, or mixed representations during inference. Experiments and user studies on a public audiobook corpus show the efficacy of our approach.