Discrete Acoustic Space for an Efficient Sampling in Neural Text-To-Speech
Not on the map. Why?
Abstract (arXiv)
We present a Split Vector Quantized Variational Autoencoder (SVQ-VAE) architecture using a split vector quantizer for NTTS, as an enhancement to the well-known Variational Autoencoder (VAE) and Vector Quantized Variational Autoencoder (VQ-VAE) architectures. Compared to these previous architectures, our proposed model retains the benefits of using an utterance-level bottleneck, while keeping significant representation power and a discretized latent space small enough for efficient prediction from text. We train the model on recordings in the expressive task-oriented dialogues domain and show that SVQ-VAE achieves a statistically significant improvement in naturalness over the VAE and VQ-VAE models. Furthermore, we demonstrate that the SVQ-VAE latent acoustic space is predictable from text, reducing the gap between the standard constant vector synthesis and vocoded recordings by 32%.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 25 of its 28 references that are in the map. First posted on arXiv 24 Oct 2021.
Builds on
25 of 28 references.
References · 25
- 2017Neural Discrete Representation Learningcited by 4,650
- 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
- 2013Auto-Encoding Variational Bayescited by 3,980
- 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,376
- 2019An Introduction to Variational Autoencoderscited by 997
- 2020A Primer in BERTology: What we know about how BERT workscited by 807
- 2017Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictionscited by 800
- 2020Jukebox: A Generative Model for Musiccited by 535
- 2015Attention-Based Models for Speech Recognitioncited by 400
- 2018Towards End-to-End Prosody Transfer for Expressive Speech Synthesis with Tacotroncited by 171
- 2018Fast Decoding in Sequence Models using Discrete Latent Variablescited by 103
- 2018Hierarchical Generative Modeling for Controllable Speech Synthesiscited by 93
13 more
- 2018Learning latent representations for style control and transfer in end-to-end speech synthesiscited by 61
- 2020Fully-hierarchical fine-grained prosody modeling for interpretable speech synthesiscited by 41
- 2020Generating diverse and natural text-to-speech samples using a quantized fine-grained VAE and auto-regressive prosody priorcited by 33
- 2021Universal Neural Vocoding with Parallel WaveNetcited by 25
- 2018Expressive Speech Synthesis via Modeling Expressions with Variational Autoencodercited by 22
- 2020CAMP: a Two-Stage Approach to Modelling Prosody in Contextcited by 12
- 2019Dynamic Prosody Generation for Speech Synthesis using Linguistics-Driven Acoustic Embedding Selectioncited by 11
- 2020Hierarchical Multi-Grained Generative Model for Expressive Speech Synthesiscited by 10
- 2020Prosodic Representation Learning and Contextual Sampling for Neural Text-to-Speechcited by 10
- 2019In Other News: A Bi-style Text-to-speech Model for Synthesizing Newscaster Voice with Limited Datacited by 7
- 2019Using generative modelling to produce varied intonation for speech synthesiscited by 7
- 2021A learned conditional prior for the VAE acoustic space of a TTS systemcited by 4
- 2021Enhancing audio quality for expressive Neural Text-to-Speechcited by 2
Built on by
Citing papers per year: 2024: 2; 2025: 1.
3 papers.