mlsanityData to 7 Oct 2026

Deep Voice 2: Multi-Speaker Neural Text-to-Speech

arXiv 1705.08947 ↗First posted 24 May 2017cs.CLcited by 75

Not on the map. Why?

Abstract (arXiv)

We introduce a technique for augmenting neural text-to-speech (TTS) with lowdimensional trainable speaker embeddings to generate different voices from a single model. As a starting point, we show improvements over the two state-ofthe-art approaches for single-speaker neural TTS: Deep Voice 1 and Tacotron. We introduce Deep Voice 2, which is based on a similar pipeline with Deep Voice 1, but constructed with higher performance building blocks and demonstrates a significant audio quality improvement over Deep Voice 1. We improve Tacotron by introducing a post-processing neural vocoder, and demonstrate a significant audio quality improvement. We then demonstrate our technique for multi-speaker speech synthesis for both Deep Voice 2 and Tacotron on two multi-speaker TTS datasets. We show that a single neural TTS system can learn hundreds of unique voices from less than half an hour of data per speaker, while achieving high audio quality synthesis and preserving the speaker identities almost perfectly.

Key facts. Cited by 75 papers in the map (data through 7 Oct 2026). First posted on arXiv 24 May 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 1; 2021: 32; 2022: 22; 2023: 10; 2024: 8; 2025: 1; 2026: 1.

75 papers.

Sort:

By area:

  1. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesiscited by 1,339
  2. 2021A Survey on Neural Speech Synthesiscited by 182
  3. 2022NaturalSpeech: End-to-End Text to Speech Synthesis with Human-Level Qualitycited by 151
  4. 2021Textless Speech-to-Speech Translation on Real Datacited by 121
  5. 2021AdaSpeech: Adaptive Text to Speech for Custom Voicecited by 98
  6. 2021Meta-StyleSpeech : Multi-Speaker Adaptive Text-to-Speech Generationcited by 93
  7. 2021Deepfakes Generation and Detection: State-of-the-art, open challenges, countermeasures, and way forwardcited by 82
  8. 2021WaveFake: A Data Set to Facilitate Audio Deepfake Detectioncited by 78
  9. 2023A Survey on Audio Diffusion Models: Text To Speech Synthesis and Enhancement in Generative AIcited by 68
  10. 2021PnG BERT: Augmented BERT on Phonemes and Graphemes for Neural TTScited by 52
  11. 2023PromptTTS 2: Describing and Generating Voices with Text Promptcited by 47
  12. 2022DiffGAN-TTS: High-Fidelity and Efficient Text-to-Speech with Denoising Diffusion GANscited by 43
38 more
  1. 2022AdaSpeech 4: Adaptive Text to Speech in Zero-Shot Scenarioscited by 41
  2. 2022An Overview of Affective Speech Synthesis and Conversion in the Deep Learning Eracited by 32
  3. 2022A Review on Methods and Applications in Multimodal Deep Learningcited by 29
  4. 2022Towards Building Text-To-Speech Systems for the Next Billion Userscited by 27
  5. 2021Recent Advances and Trends in Multimodal Deep Learning: A Reviewcited by 26
  6. 2021Universal Neural Vocoding with Parallel WaveNetcited by 25
  7. 2021fairseq S^2: A Scalable and Integrable Speech Synthesis Toolkitcited by 24
  8. 2021V2C: Visual Voice Cloningcited by 23
  9. 2021Meta-TTS: Meta-Learning for Few-Shot Speaker Adaptive Text-to-Speechcited by 22
  10. 2024The Brain's Bitter Lesson: Scaling Speech Decoding With Self-Supervised Learningcited by 21
  11. 2021GANSpeech: Adversarial Training for High-Fidelity Multi-Speaker Speech Synthesiscited by 18
  12. 2022Detecting Individual Decision-Making Style: Exploring Behavioral Stylometry in Chesscited by 18
  13. 2021SpeechNet: A Universal Modularized Model for Speech Processing Taskscited by 16
  14. 2022Low-Resource Multilingual and Zero-Shot Multispeaker TTScited by 15
  15. 2021Non-Autoregressive TTS with Explicit Duration Modelling for Low-Resource Highly Expressive Speechcited by 14
  16. 2021Speaker Generationcited by 13
  17. 2021VARA-TTS: Non-Autoregressive Text-to-Speech Synthesis based on Very Deep VAE with Residual Attentioncited by 12
  18. 2021Adapting TTS models For New Speakers using Transfer Learningcited by 12
  19. 2021Emotional Prosody Control for Speech Generationcited by 12
  20. 2021Expressive Neural Voice Cloningcited by 11
  21. 2021Style Equalization: Unsupervised Learning of Controllable Generative Sequence Modelscited by 11
  22. 2021On-device neural speech synthesiscited by 9
  23. 2022Audio representations for deep learning in sound synthesis: A reviewcited by 9
  24. 2022FCTalker: Fine and Coarse Grained Context Modeling for Expressive Conversational Speech Synthesiscited by 9
  25. 2022Cross-Speaker Emotion Transfer for Low-Resource Text-to-Speech Using Non-Parallel Voice Conversion with Pitch-Shift Data Augmentationcited by 8
  26. 2021TalkNet 2: Non-Autoregressive Depth-Wise Separable Convolutional Model for Speech Synthesis with Explicit Pitch and Duration Predictioncited by 7
  27. 2021Review of end-to-end speech synthesis technology based on deep learningcited by 6
  28. 2021How Deep Are the Fakes? Focusing on Audio Deepfake: A Surveycited by 6
  29. 2021Interactive Storytelling for Children: A Case-study of Design and Development Considerations for Ethical Conversational AIcited by 5
  30. 2022ECAPA-TDNN for Multi-speaker Text-to-speech Synthesiscited by 5
  31. 2022A Text-to-Speech Pipeline, Evaluation Methodology, and Initial Fine-Tuning Results for Child Speech Synthesiscited by 5
  32. 2022Simple and Effective Multi-sentence TTS with Expressive and Coherent Prosodycited by 5
  33. 2023Generalizable Zero-Shot Speaker Adaptive Speech Synthesis with Disentangled Representationscited by 5
  34. 2022TGAVC: Improving Autoencoder Voice Conversion with Text-Guided and Adversarial Trainingcited by 4
  35. 2023Do Prosody Transfer Models Transfer Prosody?cited by 4
  36. 2024VoxGenesis: Unsupervised Discovery of Latent Speaker Manifold for Speech Synthesiscited by 4
  37. 2024MS2SL: Multimodal Spoken Data-Driven Continuous Sign Language Productioncited by 4
  38. 2024SF-Speech: Straightened Flow for Zero-Shot Voice Clonecited by 4

50 most cited of 75 All 75 (JSON)