mlsanityData to 7 Oct 2026

Semi-Supervised Generative Modeling for Controllable Speech Synthesis

arXiv 1910.01709 ↗First posted 3 Oct 2019cs.CL cs.LG cs.SDcited by 21

Not on the map. Why?

Abstract (arXiv)

We present a novel generative model that combines state-of-the-art neural text-to-speech (TTS) with semi-supervised probabilistic latent variable models. By providing partial supervision to some of the latent variables, we are able to force them to take on consistent and interpretable purposes, which previously hasn't been possible with purely unsupervised TTS models. We demonstrate that our model is able to reliably discover and control important but rarely labelled attributes of speech, such as affect and speaking rate, with as little as 1% (30 minutes) supervision. Even at such low supervision levels we do not observe a degradation of synthesis quality compared to a state-of-the-art baseline. Audio samples are available on the web.

Key facts. Cited by 21 papers in the map (data through 7 Oct 2026). First posted on arXiv 3 Oct 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 5; 2022: 7; 2023: 3; 2024: 3; 2025: 1; 2026: 2.

21 papers.

Sort:

By area:

  1. 2021A Survey on Neural Speech Synthesiscited by 182
  2. 2023Generative Pre-training for Speech with Flow Matchingcited by 55
  3. 2024EmoSphere-TTS: Emotional Style and Intensity Modeling via Spherical Emotion Vector for Controllable Emotional Text-to-Speechcited by 23
  4. 2022Controllable Data Generation by Deep Learning: A Reviewcited by 20
  5. 2024EmoSphere++: Emotion-Controllable Zero-Shot Text-to-Speech via Emotion-Adaptive Spherical Vectorcited by 19
  6. 2023The Ethical Implications of Generative Audio Models: A Systematic Literature Reviewcited by 18
  7. 2021Speaker Generationcited by 13
  8. 2021Emotional Prosody Control for Speech Generationcited by 12
  9. 2021Prosodic Clustering for Phoneme-level Prosody Control in End-to-End Speech Synthesiscited by 8
  10. 2022Unsupervised word-level prosody tagging for controllable speech synthesiscited by 7
  11. 2022Style-Label-Free: Cross-Speaker Style Transfer by Quantized VAE and Speaker-wise Normalization in Speech Synthesiscited by 7
  12. 2023Improving Prosody for Cross-Speaker Style Transfer by Semi-Supervised Style Extractor and Hierarchical Modeling in Speech Synthesiscited by 7
9 more
  1. 2022Controllable speech synthesis by learning discrete phoneme-level prosodic representationscited by 3
  2. 2024Sound Check: Auditing Audio Datasetscited by 3
  3. 2022Text-to-speech synthesis based on latent variable conversion using diffusion probabilistic model and variational autoencodercited by 2
  4. 2025UDDETTS: Unifying Discrete and Dimensional Emotions for Controllable Emotional Text-to-Speechcited by 2
  5. 2022ReCAB-VAE: Gumbel-Softmax Variational Inference Based on Analytic Divergencecited by 1
  6. 2021Challenging the Semi-Supervised VAE Framework for Text Classificationcited by 0
  7. 2022Semi-Supervised Junction Tree Variational Autoencoder for Molecular Property Predictioncited by 0
  8. 2026Iterative Self-Learning for Expressive Text-to-Speech Synthesiscited by 0
  9. 2026Alignment-Free Text-Audiobox for Voice Dubbing and Full-Duplex Dialogue Synthesiscited by 0