mlsanityData to 14 Oct 2026

Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictions

arXiv 1712.05884 ↗First posted 16 Dec 2017cs.CLcited by 804

Not on the map. Why?

Cited by 68 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

This paper describes Tacotron 2, a neural network architecture for speech synthesis directly from text. The system is composed of a recurrent sequence-to-sequence feature prediction network that maps character embeddings to mel-scale spectrograms, followed by a modified WaveNet model acting as a vocoder to synthesize timedomain waveforms from those spectrograms. Our model achieves a mean opinion score (MOS) of 4.53 comparable to a MOS of 4.58 for professionally recorded speech. To validate our design choices, we present ablation studies of key components of our system and evaluate the impact of using mel spectrograms as the input to WaveNet instead of linguistic, duration, and F0 features. We further demonstrate that using a compact acoustic intermediate representation enables significant simplification of the WaveNet architecture.

Key facts. Cited by 804 papers in the map (data through 14 Oct 2026). First posted on arXiv 16 Dec 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2020: 2; 2021: 170; 2022: 188; 2023: 163; 2024: 133; 2025: 84; 2026: 64.

804 papers.

Sort:

By area:

  1. 2020GShard: Scaling Giant Models with Conditional Computation and Automatic Shardingcited by 1,631
  2. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesiscited by 1,354
  3. 2023Neural Codec Language Models are Zero-Shot Text to Speech Synthesizerscited by 752
  4. 2021Grad-TTS: A Diffusion Probabilistic Model for Text-to-Speechcited by 447
  5. 2023High-Fidelity Audio Compression with Improved RVQGANcited by 431
  6. 2023Voicebox: Text-Guided Multilingual Universal Speech Generation at Scalecited by 379
  7. 2024F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matchingcited by 361
  8. 2021Generative Spoken Language Modeling from Raw Audiocited by 340
  9. 2024CosyVoice 2: Scalable Streaming Speech Synthesis with Large Language Modelscited by 288
  10. 2021YourTTS: Towards Zero-Shot Multi-Speaker TTS and Zero-Shot Voice Conversion for everyonecited by 278
  11. 2022Self-Supervised Speech Representation Learning: A Reviewcited by 260
  12. 2023NaturalSpeech 2: Latent Diffusion Models are Natural and Zero-Shot Speech and Singing Synthesizerscited by 235
38 more
  1. 2024NaturalSpeech 3: Zero-Shot Speech Synthesis with Factorized Codec and Diffusion Modelscited by 226
  2. 2022MoFusion: A Framework for Denoising-Diffusion-based Motion Synthesiscited by 218
  3. 2021A Survey on Neural Speech Synthesiscited by 184
  4. 2021SpeechT5: Unified-Modal Encoder-Decoder Pre-Training for Spoken Language Processingcited by 180
  5. 2024XTTS: a Massively Multilingual Zero-Shot Text-to-Speech Modelcited by 179
  6. 2024LLaMA-Omni: Seamless Speech Interaction with Large Language Modelscited by 173
  7. 2022It's Raw! Audio Generation with State-Space Modelscited by 171
  8. 2021Diff-TTS: A Denoising Diffusion Model for Text-to-Speechcited by 157
  9. 2021DiffSinger: Singing Voice Synthesis via Shallow Diffusion Mechanismcited by 154
  10. 2023Vocos: Closing the gap between time-domain and Fourier-based neural vocoders for high-quality audio synthesiscited by 153
  11. 2022NaturalSpeech: End-to-End Text to Speech Synthesis with Human-Level Qualitycited by 152
  12. 2025CosyVoice 3: Towards In-the-wild Speech Generation via Scaling-up and Post-trainingcited by 150
  13. 2022ProDiff: Progressive Fast Diffusion Model For High-Quality Text-to-Speechcited by 149
  14. 2024VALL-E 2: Neural Codec Language Models are Human Parity Zero-Shot Text to Speech Synthesizerscited by 122
  15. 2021Textless Speech-to-Speech Translation on Real Datacited by 121
  16. 2023Audiobox: Unified Audio Generation with Natural Language Promptscited by 120
  17. 2025On The Landscape of Spoken Language Models: A Comprehensive Surveycited by 118
  18. 2025Step-Audio 2 Technical Reportcited by 105
  19. 2023Textually Pretrained Speech Language Modelscited by 101
  20. 2021AdaSpeech: Adaptive Text to Speech for Custom Voicecited by 98
  21. 2022ADD 2022: the First Audio Deep Synthesis Detection Challengecited by 96
  22. 2023Spoken Question Answering and Speech Continuation Using Spectrogram-Powered LLMcited by 94
  23. 2021Meta-StyleSpeech : Multi-Speaker Adaptive Text-to-Speech Generationcited by 93
  24. 2024ASVspoof 5: Crowdsourced Speech Data, Deepfakes, and Adversarial Attacks at Scalecited by 91
  25. 2025IndexTTS2: A Breakthrough in Emotionally Expressive and Duration-Controlled Auto-Regressive Zero-Shot Text-to-Speechcited by 90
  26. 2023VioLA: Unified Codec Language Models for Speech Recognition, Synthesis, and Translationcited by 87
  27. 2024Recent Advances in Speech Language Models: A Surveycited by 87
  28. 2021Guided-TTS: A Diffusion Model for Text-to-Speech via Classifier Guidancecited by 84
  29. 2023A Complete Survey on Generative AI (AIGC): Is ChatGPT from GPT-4 to GPT-5 All You Need?cited by 84
  30. 2023Temporal Dynamic Quantization for Diffusion Modelscited by 81
  31. 2021WaveFake: A Data Set to Facilitate Audio Deepfake Detectioncited by 78
  32. 2022SUPERB-SG: Enhanced Speech processing Universal PERformance Benchmark for Semantic and Generative Capabilitiescited by 77
  33. 2024Detecting Multimedia Generated by Large AI Models: A Surveycited by 76
  34. 2024BASE TTS: Lessons from building a billion-parameter Text-to-Speech model on 100K hours of datacited by 75
  35. 2023Voxtlm: unified decoder-only models for consolidating speech recognition/synthesis and speech/text continuation taskscited by 72
  36. 2021Translatotron 2: High-quality direct speech-to-speech translation with voice preservationcited by 71
  37. 2022DailyTalk: Spoken Dialogue Dataset for Conversational Text-to-Speechcited by 69
  38. 2023TFMQ-DM: Temporal Feature Maintenance Quantization for Diffusion Modelscited by 69

50 most cited of 804 All 804 (JSON)