mlsanityData to 14 Oct 2026

Referee: Towards reference-free cross-speaker style transfer with low-quality data for expressive speech synthesis

arXiv 2109.03439 ↗First posted 8 Sep 2021eess.AS cs.CL cs.SDcited by 0

Not on the map. Why?

Abstract (arXiv)

Cross-speaker style transfer (CSST) in text-to-speech (TTS) synthesis aims at transferring a speaking style to the synthesised speech in a target speaker's voice. Most previous CSST approaches rely on expensive high-quality data carrying desired speaking style during training and require a reference utterance to obtain speaking style descriptors as conditioning on the generation of a new sentence. This work presents Referee, a robust reference-free CSST approach for expressive TTS, which fully leverages low-quality data to learn speaking styles from text. Referee is built by cascading a text-to-style (T2S) model with a style-to-wave (S2W) model. Phonetic PosteriorGram (PPG), phoneme-level pitch and energy contours are adopted as fine-grained speaking style descriptors, which are predicted from text using the T2S model. A novel pretrain-refinement method is adopted to learn a robust T2S model by only using readily accessible low-quality data. The S2W model is trained with high-quality target data, which is adopted to effectively aggregate style descriptors and generate high-fidelity speech in the target speaker's voice. Experimental results are presented, showing that Referee outperforms a global-style-token (GST)-based baseline approach in CSST.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 18 of its 22 references that are in the map. First posted on arXiv 8 Sep 2021.

Builds on

18 of 22 references.

References · 18

  1. 2015Domain-Adversarial Training of Neural Networkscited by 3,044
  2. 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,154
  3. 2016Least Squares Generative Adversarial Networkscited by 1,001
  4. 2017Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictionscited by 804
  5. 2020FastSpeech 2: Fast and High-Quality End-to-End Text to Speechcited by 717
  6. 2017Tacotron: Towards End-to-End Speech Synthesiscited by 490
  7. 2018Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesiscited by 259
  8. 2018Towards End-to-End Prosody Transfer for Expressive Speech Synthesis with Tacotroncited by 173
  9. 2021AdaSpeech: Adaptive Text to Speech for Custom Voicecited by 98
  10. 2018Hierarchical Generative Modeling for Controllable Speech Synthesiscited by 93
  11. 2021Meta-StyleSpeech : Multi-Speaker Adaptive Text-to-Speech Generationcited by 93
  12. 2018Robust and fine-grained prosody control of end-to-end speech synthesiscited by 48
6 more
  1. 2019DurIAN: Duration Informed Attention Network For Multimodal Synthesiscited by 33
  2. 2019Fine-grained robust prosody transfer for single-speaker neural text-to-speechcited by 27
  3. 2020CopyCat: Many-to-Many Fine-Grained Prosody Transfer for Neural Text-to-Speechcited by 22
  4. 2021Glow-WaveGAN: Learning Speech Representations from GAN-based Variational Auto-Encoder For High Fidelity Flow-based Speech Synthesiscited by 10
  5. 2021Cross-speaker Style Transfer with Prosody Bottleneck in Neural Speech Synthesiscited by 9
  6. 2019DFSMN-SAN with Persistent Memory Model for Automatic Speech Recognitioncited by 4

Built on by

Not cited yet.