mlsanityData to 7 Oct 2026

A Survey on Neural Speech Synthesis

arXiv 2106.15561 ↗First posted 29 Jun 2021eess.AS cs.CL cs.LGcited by 182

Not on the map. Why?

Abstract (arXiv)

Text to speech (TTS), or speech synthesis, which aims to synthesize intelligible and natural speech given text, is a hot research topic in speech, language, and machine learning communities and has broad applications in the industry. As the development of deep learning and artificial intelligence, neural network-based TTS has significantly improved the quality of synthesized speech in recent years. In this paper, we conduct a comprehensive survey on neural TTS, aiming to provide a good understanding of current research and future trends. We focus on the key components in neural TTS, including text analysis, acoustic models and vocoders, and several advanced topics, including fast TTS, low-resource TTS, robust TTS, expressive TTS, and adaptive TTS, etc. We further summarize resources related to TTS (e.g., datasets, opensource implementations) and discuss future research directions. This survey can serve both academic researchers and industry practitioners working on TTS.

Key facts. Cited by 182 papers in the map (data through 7 Oct 2026). Builds on 266 of its 440 references that are in the map. First posted on arXiv 29 Jun 2021.

Builds on

266 of 440 references.

References · 266

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  4. 2020Denoising Diffusion Implicit Modelscited by 9,655
  5. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  6. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  7. 2013Auto-Encoding Variational Bayescited by 3,980
  8. 2017Improved Training of Wasserstein GANscited by 2,223
  9. 2015Variational Inference with Normalizing Flowscited by 2,161
  10. 2015Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networkscited by 2,048
  11. 2016Density estimation using Real NVPcited by 2,029
  12. 2016WaveNet: A Generative Model for Raw Audiocited by 1,710
38 more
  1. 2018Glow: Generative Flow with Invertible 1x1 Convolutionscited by 1,539
  2. 2016Neural Architecture Search with Reinforcement Learningcited by 1,489
  3. 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,358
  4. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesiscited by 1,339
  5. 2019Normalizing Flows for Probabilistic Modeling and Inferencecited by 1,315
  6. 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,139
  7. 2014NICE: Non-linear Independent Components Estimationcited by 1,052
  8. 2016Least Squares Generative Adversarial Networkscited by 1,000
  9. 2017Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictionscited by 800
  10. 2015Scheduled Sampling for Sequence Prediction with Recurrent Neural Networkscited by 779
  11. 2016Sequence-Level Knowledge Distillationcited by 761
  12. 2020FastSpeech 2: Fast and High-Quality End-to-End Text to Speechcited by 710
  13. 2017Masked Autoregressive Flow for Density Estimationcited by 680
  14. 2019LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speechcited by 639
  15. 2016Improving Variational Inference with Inverse Autoregressive Flowcited by 636
  16. 2020WaveGrad: Estimating Gradients for Waveform Generationcited by 620
  17. 2021Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speechcited by 555
  18. 2013Generating Sequences With Recurrent Neural Networkscited by 549
  19. 2015Listen, Attend and Spellcited by 542
  20. 2017Tacotron: Towards End-to-End Speech Synthesiscited by 487
  21. 2020MLS: A Large-Scale Multilingual Dataset for Speech Researchcited by 467
  22. 2021Grad-TTS: A Diffusion Probabilistic Model for Text-to-Speechcited by 445
  23. 2015Autoencoding beyond pixels using a learned similarity metriccited by 407
  24. 2015Attention-Based Models for Speech Recognitioncited by 400
  25. 2016SeqGAN: Sequence Generative Adversarial Nets with Policy Gradientcited by 375
  26. 2017AISHELL-1: An Open-Source Mandarin Speech Corpus and A Speech Recognition Baselinecited by 364
  27. 2019MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesiscited by 351
  28. 2019Parallel WaveGAN: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogramcited by 337
  29. 2018WaveGlow: A Flow-based Generative Network for Speech Synthesiscited by 308
  30. 2017Toward Multimodal Image-to-Image Translationcited by 295
  31. 2018Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesiscited by 280
  32. 2020Glow-TTS: A Generative Flow for Text-to-Speech via Monotonic Alignment Searchcited by 269
  33. 2018Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesiscited by 257
  34. 2021Speech Resynthesis from Discrete Disentangled Self-Supervised Representationscited by 243
  35. 2018Adversarial Audio Synthesiscited by 222
  36. 2018Efficient Neural Audio Synthesiscited by 218
  37. 2016Professor Forcing: A New Algorithm for Training Recurrent Networkscited by 214
  38. 2017Parallel WaveNet: Fast High-Fidelity Speech Synthesiscited by 214

50 most cited of 266 All 266 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 12; 2022: 39; 2023: 42; 2024: 48; 2025: 25; 2026: 16.

182 papers.

Sort:

By area:

  1. 2023Neural Codec Language Models are Zero-Shot Text to Speech Synthesizerscited by 737
  2. 2023Simple and Controllable Music Generationcited by 480
  3. 2023Scaling Speech Technology to 1,000+ Languagescited by 401
  4. 2021YourTTS: Towards Zero-Shot Multi-Speaker TTS and Zero-Shot Voice Conversion for everyonecited by 277
  5. 2022Diffsound: Discrete Diffusion Model for Text-to-sound Generationcited by 256
  6. 2023NaturalSpeech 2: Latent Diffusion Models are Natural and Zero-Shot Speech and Singing Synthesizerscited by 233
  7. 2024Emilia: An Extensive, Multilingual, and Diverse Speech Dataset for Large-Scale Speech Generationcited by 200
  8. 2024XTTS: a Massively Multilingual Zero-Shot Text-to-Speech Modelcited by 174
  9. 2022NaturalSpeech: End-to-End Text to Speech Synthesis with Human-Level Qualitycited by 151
  10. 2023StyleTTS 2: Towards Human-Level Text-to-Speech through Style Diffusion and Adversarial Training with Large Speech Language Modelscited by 148
  11. 2022PromptTTS: Controllable Text-to-Speech with Text Descriptionscited by 106
  12. 2024BASE TTS: Lessons from building a billion-parameter Text-to-Speech model on 100K hours of datacited by 74
38 more
  1. 2023A Survey on Audio Diffusion Models: Text To Speech Synthesis and Enhancement in Generative AIcited by 68
  2. 2023A Comprehensive Review of Data-Driven Co-Speech Gesture Generationcited by 66
  3. 2024EARS: An Anechoic Fullband Speech Dataset Benchmarked for Speech Enhancement and Dereverberationcited by 62
  4. 2023Mega-TTS: Zero-Shot Text-to-Speech at Scale with Intrinsic Inductive Biascited by 51
  5. 2023PromptTTS 2: Describing and Generating Voices with Text Promptcited by 47
  6. 2022BinauralGrad: A Two-Stage Conditional Diffusion Probabilistic Model for Binaural Audio Synthesiscited by 44
  7. 2023CoMoSpeech: One-Step Speech and Singing Voice Synthesis via Consistency Modelcited by 44
  8. 2022AdaSpeech 4: Adaptive Text to Speech in Zero-Shot Scenarioscited by 41
  9. 2025MegaTTS 3: Sparse Alignment Enhanced Latent Diffusion Transformer for Zero-Shot Speech Synthesiscited by 39
  10. 2023Transformers in Speech Processing: A Surveycited by 38
  11. 2024VALL-E R: Robust and Efficient Zero-Shot Text-to-Speech Synthesis via Monotonic Alignmentcited by 37
  12. 2022StyleTTS: A Style-Based Generative Model for Natural and Diverse Text-to-Speech Synthesiscited by 32
  13. 2022An Overview of Affective Speech Synthesis and Conversion in the Deep Learning Eracited by 32
  14. 2021Textless Speech Emotion Conversion using Discrete and Decomposed Representationscited by 29
  15. 2023Schrodinger Bridges Beat Diffusion Models on Text-to-Speech Synthesiscited by 29
  16. 2022InferGrad: Improving Diffusion Models for Vocoder by Considering Inference in Trainingcited by 28
  17. 2022Speech Synthesis with Mixed Emotionscited by 28
  18. 2022Towards Building Text-To-Speech Systems for the Next Billion Userscited by 27
  19. 2023PromptTTS++: Controlling Speaker Identity in Prompt-Based Text-to-Speech Using Natural Language Descriptionscited by 25
  20. 2021fairseq S^2: A Scalable and Integrable Speech Synthesis Toolkitcited by 24
  21. 2023Phoneme-Level BERT for Enhanced Prosody of Text-to-Speech with Grapheme Predictionscited by 22
  22. 2023From Discrete Tokens to High-Fidelity Audio Using Multi-Band Diffusioncited by 22
  23. 2024FlashSpeech: Efficient Zero-Shot Speech Synthesiscited by 22
  24. 2024Towards Controllable Speech Synthesis in the Era of Large Language Models: A Systematic Surveycited by 22
  25. 2022ResGrad: Residual Denoising Diffusion Probabilistic Models for Text to Speechcited by 21
  26. 2024Preference Alignment Improves Language Model-Based TTScited by 21
  27. 2022Controllable Data Generation by Deep Learning: A Reviewcited by 20
  28. 2022VideoDubber: Machine Translation with Speech-Aware Length Control for Video Dubbingcited by 20
  29. 2023Bias and Fairness in Chatbots: An Overviewcited by 20
  30. 2024Leveraging Large Language Model as Simulated Patients for Clinical Educationcited by 20
  31. 2024SafeEar: Content Privacy-Preserving Audio Deepfake Detectioncited by 20
  32. 2021DelightfulTTS: The Microsoft Speech Synthesis System for Blizzard Challenge 2021cited by 19
  33. 2025Emilia: A Large-Scale, Extensive, Multilingual, and Diverse Dataset for Speech Generationcited by 19
  34. 2022WavThruVec: Latent speech representation as intermediate features for neural speech synthesiscited by 17
  35. 2023The Age of Synthetic Realities: Challenges and Opportunitiescited by 15
  36. 2024CoVoMix: Advancing Zero-Shot Speech Generation for Human-like Multi-talker Conversationscited by 15
  37. 2025NVSpeech: An Integrated and Scalable Pipeline for Human-Like Speech Modeling with Paralinguistic Vocalizationscited by 14
  38. 2022Mixed-Phoneme BERT: Improving BERT with Mixed Phoneme and Sup-Phoneme Representations for Text to Speechcited by 13

50 most cited of 182 All 182 (JSON)