Enhancing audio quality for expressive Neural Text-to-Speech
Not on the map. Why?
Abstract (arXiv)
Artificial speech synthesis has made a great leap in terms of naturalness as recent Text-to-Speech (TTS) systems are capable of producing speech with similar quality to human recordings. However, not all speaking styles are easy to model: highly expressive voices are still challenging even to recent TTS architectures since there seems to be a trade-off between expressiveness in a generated audio and its signal quality. In this paper, we present a set of techniques that can be leveraged to enhance the signal quality of a highly-expressive voice without the use of additional data. The proposed techniques include: tuning the autoregressive loop's granularity during training; using Generative Adversarial Networks in acoustic modelling; and the use of Variational Auto-Encoders in both the acoustic model and the neural vocoder. We show that, when combined, these techniques greatly closed the gap in perceived naturalness between the baseline system and recordings by 39% in terms of MUSHRA scores for an expressive celebrity voice.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 27 of its 35 references that are in the map. First posted on arXiv 13 Aug 2021.
Builds on
27 of 35 references.
References · 27
- 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
- 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
- 2016WaveNet: A Generative Model for Raw Audiocited by 1,710
- 2018Spectral Normalization for Generative Adversarial Networkscited by 1,618
- 2018Self-Attention Generative Adversarial Networkscited by 809
- 2017Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictionscited by 800
- 2016Improving Variational Inference with Inverse Autoregressive Flowcited by 636
- 2015Autoencoding beyond pixels using a learned similarity metriccited by 407
- 2015Attention-Based Models for Speech Recognitioncited by 400
- 2018Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesiscited by 257
- 2017Learning Discourse-level Diversity for Neural Dialog Models using Conditional Variational Autoencoderscited by 222
- 2015Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networkscited by 214
15 more
- 2017Parallel WaveNet: Fast High-Fidelity Speech Synthesiscited by 214
- 2017SEGAN: Speech Enhancement Generative Adversarial Networkcited by 165
- 2018Hierarchical Generative Modeling for Controllable Speech Synthesiscited by 93
- 2017Online and Linear-Time Attention by Enforcing Monotonic Alignmentscited by 85
- 2018Semi-Amortized Variational Autoencoderscited by 83
- 2017Improved Variational Autoencoders for Text Modeling using Dilated Convolutionscited by 81
- 2018Avoiding Latent Variable Collapse With Generative Skip Modelscited by 47
- 2019Location-Relative Attention Mechanisms For Robust Long-Form Speech Synthesiscited by 44
- 2019Robust Sequence-to-Sequence Acoustic Modeling with Stepwise Monotonic Attention for Neural TTScited by 33
- 2018Forward Attention in Sequence-to-sequence Acoustic Modelling for Speech Synthesiscited by 17
- 2018Effect of data reduction on sequence-to-sequence neural TTScited by 16
- 2019A New GAN-based End-to-End TTS Training Algorithmcited by 13
- 2015Unsupervised Learning of Visual Structure using Predictive Generative Networkscited by 12
- 2019In Other News: A Bi-style Text-to-speech Model for Synthesizing Newscaster Voice with Limited Datacited by 7
- 2020Parallel WaveNet conditioned on VAE latent vectorscited by 3
Built on by
Citing papers per year: 2021: 1; 2022: 0; 2023: 1.
2 papers.
- 2023Creating New Voices using Normalizing Flows22 Dec 2023
- 2021Discrete Acoustic Space for an Efficient Sampling in Neural Text-To-Speech24 Oct 2021