UniTTS: Residual Learning of Unified Embedding Space for Speech Style Control
Not on the map. Why?
Abstract (arXiv)
We propose a novel high-fidelity expressive speech synthesis model, UniTTS, that learns and controls overlapping style attributes avoiding interference. UniTTS represents multiple style attributes in a single unified embedding space by the residuals between the phoneme embeddings before and after applying the attributes. The proposed method is especially effective in controlling multiple attributes that are difficult to separate cleanly, such as speaker ID and emotion, because it minimizes redundancy when adding variance in speaker ID and emotion, and additionally, predicts duration, pitch, and energy based on the speaker ID and emotion. In experiments, the visualization results exhibit that the proposed methods learned multiple attributes harmoniously in a manner that can be easily separated again. As well, UniTTS synthesized high-fidelity speech signals controlling multiple style attributes. The synthesized speech samples are presented at https://anonymous-authors2022.github.io/paper_works/UniTTS/demos/.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 27 of its 38 references that are in the map. First posted on arXiv 21 Jun 2021.
Builds on
27 of 38 references.
References · 27
- 2017Attention Is All You Needcited by 59,932
- 2015Domain-Adversarial Training of Neural Networkscited by 3,023
- 2016WaveNet: A Generative Model for Raw Audiocited by 1,710
- 2017Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictionscited by 800
- 2020FastSpeech 2: Fast and High-Quality End-to-End Text to Speechcited by 710
- 2017Tacotron: Towards End-to-End Speech Synthesiscited by 487
- 2019Parallel WaveGAN: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogramcited by 337
- 2018WaveGlow: A Flow-based Generative Network for Speech Synthesiscited by 308
- 2018Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesiscited by 257
- 2017Parallel WaveNet: Fast High-Fidelity Speech Synthesiscited by 214
- 2018Towards End-to-End Prosody Transfer for Expressive Speech Synthesis with Tacotroncited by 171
- 2020FastPitch: Parallel Text-to-speech with Pitch Predictioncited by 150
15 more
- 2018Hierarchical Generative Modeling for Controllable Speech Synthesiscited by 93
- 2020Unsupervised Speech Decomposition via Triple Information Bottleneckcited by 87
- 2017Deep Voice 2: Multi-Speaker Neural Text-to-Speechcited by 75
- 2017Efficiently Trainable Text-to-Speech System Based on Deep Convolutional Networks with Guided Attentioncited by 57
- 2018Robust and fine-grained prosody control of end-to-end speech synthesiscited by 48
- 2020Fully-hierarchical fine-grained prosody modeling for interpretable speech synthesiscited by 41
- 2020HiFi-GAN: High-Fidelity Denoising and Dereverberation Based on Speech Deep Features in Adversarial Networkscited by 40
- 2020VocGAN: A High-Fidelity Real-time Vocoder with a Hierarchically-nested Adversarial Networkcited by 26
- 2019CHiVE: Varying Prosody in Speech Synthesis with a Linguistically Driven Dynamic Hierarchical Conditional Variational Networkcited by 25
- 2020AlignTTS: Efficient Feed-Forward Text-to-Speech System without Explicit Alignmentcited by 20
- 2019End-to-End Emotional Speech Synthesis Using Style Tokens and Semi-Supervised Trainingcited by 19
- 2020Hierarchical Prosody Modeling for Non-Autoregressive Speech Synthesiscited by 16
- 2021VARA-TTS: Non-Autoregressive Text-to-Speech Synthesis based on Very Deep VAE with Residual Attentioncited by 12
- 2019A Methodology for Controlling the Emotional Expressiveness in Synthetic Speech -- a Deep Learning approachcited by 3
- 2020Disentangled speaker and nuisance attribute embedding for robust speaker verificationcited by 2
Built on by
Citing papers per year: 2022: 1.
1 paper.