# Controllable and Lossless Non-Autoregressive End-to-End Text-to-Speech

arXiv: https://arxiv.org/abs/2207.06088 · HTML: https://mlsanity.com/paper/2207.06088/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Jul 2022.

## Built on by (most cited first, 3 of 3)

- [Towards Controllable Speech Synthesis in the Era of Large Language Models: A Systematic Survey](https://mlsanity.com/paper/2412.06602/) (2024)
- [Controllable Accented Text-to-Speech Synthesis](https://mlsanity.com/paper/2209.10804/) (2022)
- [Investigating Disentanglement in a Phoneme-level Speech Codec for Prosody Modeling](https://mlsanity.com/paper/2409.08664/) (2024)

## Abstract (arXiv)

Some recent studies have demonstrated the feasibility of single-stage neural text-to-speech, which does not need to generate mel-spectrograms but generates the raw waveforms directly from the text. Single-stage text-to-speech often faces two problems: a) the one-to-many mapping problem due to multiple speech variations and b) insufficiency of high frequency reconstruction due to the lack of supervision of ground-truth acoustic features during training. To solve the a) problem and generate more expressive speech, we propose a novel phoneme-level prosody modeling method based on a variational autoencoder with normalizing flows to model underlying prosodic information in speech. We also use the prosody predictor to support end-to-end expressive speech synthesis. Furthermore, we propose the dual parallel autoencoder to introduce supervision of the ground-truth acoustic features during training to solve the b) problem enabling our model to generate high-quality speech. We compare the synthesis quality with state-of-the-art text-to-speech systems on an internal expressive English dataset. Both qualitative and quantitative evaluations demonstrate the superiority and robustness of our method for lossless speech generation while also showing a strong capability in prosody modeling.