Cross-Utterance Conditioned VAE for Non-Autoregressive Text-to-Speech
Not on the map. Why?
Abstract (arXiv)
Modelling prosody variation is critical for synthesizing natural and expressive speech in end-to-end text-to-speech (TTS) systems. In this paper, a cross-utterance conditional VAE (CUC-VAE) is proposed to estimate a posterior probability distribution of the latent prosody features for each phoneme by conditioning on acoustic features, speaker information, and text features obtained from both past and future sentences. At inference time, instead of the standard Gaussian distribution used by VAE, CUC-VAE allows sampling from an utterance-specific prior distribution conditioned on cross-utterance information, which allows the prosody features generated by the TTS system to be related to the context and is more similar to how humans naturally produce prosody. The performance of CUC-VAE is evaluated via a qualitative listening test for naturalness, intelligibility and quantitative measurements, including word error rates and the standard deviation of prosody attributes. Experimental results on LJ-Speech and LibriTTS data show that the proposed CUC-VAE TTS system improves naturalness and prosody diversity with clear margins.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 26 of its 37 references that are in the map. First posted on arXiv 9 May 2022.
Builds on
26 of 37 references.
References · 26
- 2017Attention Is All You Needcited by 60,361
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2016WaveNet: A Generative Model for Raw Audiocited by 1,722
- 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,154
- 2017Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictionscited by 804
- 2020FastSpeech 2: Fast and High-Quality End-to-End Text to Speechcited by 717
- 2019LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speechcited by 649
- 2017Tacotron: Towards End-to-End Speech Synthesiscited by 490
- 2020FastPitch: Parallel Text-to-speech with Pitch Predictioncited by 151
- 2018Hierarchical Generative Modeling for Controllable Speech Synthesiscited by 93
- 2021PnG BERT: Augmented BERT on Phonemes and Graphemes for Neural TTScited by 52
- 2018Robust and fine-grained prosody control of end-to-end speech synthesiscited by 48
14 more
- 2020Fully-hierarchical fine-grained prosody modeling for interpretable speech synthesiscited by 41
- 2020Generating diverse and natural text-to-speech samples using a quantized fine-grained VAE and auto-regressive prosody priorcited by 33
- 2018Predicting Expressive Speaking Style From Text In End-To-End Speech Synthesiscited by 32
- 2021Parallel Tacotron 2: A Non-Autoregressive Neural TTS Model with Differentiable Duration Modelingcited by 31
- 2020Improving Prosody Modelling with Cross-Utterance BERT Embeddings for End-to-end Speech Synthesiscited by 27
- 2018Expressive Speech Synthesis via Modeling Expressions with Variational Autoencodercited by 22
- 2020AlignTTS: Efficient Feed-Forward Text-to-Speech System without Explicit Alignmentcited by 20
- 2021Controllable Context-aware Conversational Speech Synthesiscited by 18
- 2021Phrase break prediction with bidirectional encoder representations in Japanese text-to-speech synthesiscited by 12
- 2021Speech BERT Embedding For Improving Prosody in Neural TTScited by 11
- 2019Towards Transfer Learning for End-to-End Speech Synthesis from Deep Pre-Trained Language Modelscited by 10
- 2019Non-Autoregressive Neural Text-to-Speechcited by 9
- 2020End-to-End Text-to-Speech using Latent Duration based on VQ-VAEcited by 7
- 2021A learned conditional prior for the VAE acoustic space of a TTS systemcited by 4
Built on by
Citing papers per year: 2023: 1.
1 paper.
- 2023Cross-Utterance Conditioned VAE for Speech Generationcited by 1
- 2023Cross-Utterance Conditioned VAE for Speech Generation8 Sep 2023