mlsanityData to 7 Oct 2026

Speaker Generation

arXiv 2111.05095 ↗First posted 7 Nov 2021cs.SD cs.CL cs.LGcited by 13

Not on the map. Why?

Abstract (arXiv)

This work explores the task of synthesizing speech in nonexistent human-sounding voices. We call this task "speaker generation", and present TacoSpawn, a system that performs competitively at this task. TacoSpawn is a recurrent attention-based text-to-speech model that learns a distribution over a speaker embedding space, which enables sampling of novel and diverse speakers. Our method is easy to implement, and does not require transfer learning from speaker ID systems. We present objective and subjective metrics for evaluating performance on this task, and demonstrate that our proposed objective metrics correlate with human perception of speaker similarity. Audio samples are available on our demo page.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 28 of its 40 references that are in the map. First posted on arXiv 7 Nov 2021.

Builds on

28 of 40 references.

References · 28

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2016WaveNet: A Generative Model for Raw Audiocited by 1,710
  3. 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,139
  4. 2017Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictionscited by 800
  5. 2013Generating Sequences With Recurrent Neural Networkscited by 549
  6. 2016f-GAN: Training Generative Neural Samplers using Variational Divergence Minimizationcited by 504
  7. 2017Tacotron: Towards End-to-End Speech Synthesiscited by 487
  8. 2019Parallel WaveGAN: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogramcited by 337
  9. 2018Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesiscited by 280
  10. 2018Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesiscited by 257
  11. 2018Efficient Neural Audio Synthesiscited by 218
  12. 2018Towards End-to-End Prosody Transfer for Expressive Speech Synthesis with Tacotroncited by 171
16 more
  1. 2018Neural Voice Cloning with a Few Samplescited by 100
  2. 2021AdaSpeech: Adaptive Text to Speech for Custom Voicecited by 98
  3. 2018Taming VAEscited by 85
  4. 2019One-shot Voice Conversion by Separating Speaker and Content Representations with Instance Normalizationcited by 82
  5. 2017Deep Voice 2: Multi-Speaker Neural Text-to-Speechcited by 75
  6. 2019Zero-Shot Multi-Speaker Text-To-Speech with State-of-the-art Neural Speaker Embeddingscited by 73
  7. 2021SC-GlowTTS: an Efficient Zero-Shot Multi-Speaker Text-To-Speech Modelcited by 56
  8. 2019Location-Relative Attention Mechanisms For Robust Long-Form Speech Synthesiscited by 44
  9. 2021Investigating on Incorporating Pretrained and Learnable Speaker Representations for Multi-Speaker Multi-Style Text-to-Speechcited by 37
  10. 2018Predicting Expressive Speaking Style From Text In End-To-End Speech Synthesiscited by 31
  11. 2020Attentron: Few-Shot Text-to-Speech Utilizing Attention-Based Variable-Length Embeddingcited by 27
  12. 2020BOFFIN TTS: Few-Shot Speaker Adaptation by Bayesian Optimizationcited by 22
  13. 2019Semi-Supervised Generative Modeling for Controllable Speech Synthesiscited by 21
  14. 2019Effective Use of Variational Embedding Capacity in Expressive End-to-End Speech Synthesiscited by 15
  15. 2016Improved generator objectives for GANscited by 12
  16. 2020Properties of f-divergences and f-GAN trainingcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 6; 2023: 4; 2024: 1; 2025: 1; 2026: 1.

13 papers.

Sort:

  1. 2023PromptTTS++: Controlling Speaker Identity in Prompt-Based Text-to-Speech Using Natural Language Descriptionscited by 25
  2. 2023Text-only domain adaptation for end-to-end ASR using integrated text-to-mel-spectrogram generatorcited by 11
  3. 2022Evaluating and reducing the distance between synthetic and real speech distributionscited by 8
  4. 2024VoxGenesis: Unsupervised Discovery of Latent Speaker Manifold for Speech Synthesiscited by 4
  5. 2022Stutter-TTS: Controlled Synthesis and Improved Recognition of Stuttered Speechcited by 3
  6. 2022Self-supervised learning for robust voice cloningcited by 2
  7. 2023Residual Information in Deep Speaker Embedding Architecturescited by 2
  8. 2022Multi-Task Adversarial Training Algorithm for Multi-Speaker Neural Text-to-Speechcited by 1
  9. 2023Creating New Voices using Normalizing Flowscited by 1
  10. 2022R-MelNet: Reduced Mel-Spectral Modeling for Neural TTScited by 0
  11. 2022Generating Multilingual Gender-Ambiguous Text-to-Speech Voicescited by 0
  12. 2025SequenceLayers: Sequence Processing and Streaming Neural Networks Made Easycited by 0
1 more
  1. 2026NouveauVoice: Generating Novel Pseudo Speakers for Voice Anonymizationcited by 0