mlsanityData to 7 Oct 2026

StyleTTS: A Style-Based Generative Model for Natural and Diverse Text-to-Speech Synthesis

arXiv 2205.15439 ↗First posted 30 May 2022eess.AS cs.CL cs.LGcited by 32

Not on the map. Why?

Abstract (arXiv)

Text-to-Speech (TTS) has recently seen great progress in synthesizing high-quality speech owing to the rapid development of parallel TTS systems, but producing speech with naturalistic prosodic variations, speaking styles and emotional tones remains challenging. Moreover, since duration and speech are generated separately, parallel TTS models still have problems finding the best monotonic alignments that are crucial for naturalistic speech synthesis. Here, we propose StyleTTS, a style-based generative model for parallel TTS that can synthesize diverse speech with natural prosody from a reference speech utterance. With novel Transferable Monotonic Aligner (TMA) and duration-invariant data augmentation schemes, our method significantly outperforms state-of-the-art models on both single and multi-speaker datasets in subjective tests of speech naturalness and speaker similarity. Through self-supervised learning of the speaking styles, our model can synthesize speech with the same prosodic and emotional tone as any given reference speech without the need for explicitly labeling these categories.

Key facts. Cited by 32 papers in the map (data through 7 Oct 2026). Builds on 54 of its 83 references that are in the map. First posted on arXiv 30 May 2022.

Builds on

54 of 83 references.

References · 54

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  3. 2016Layer Normalizationcited by 5,579
  4. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  5. 2016Improved Techniques for Training GANscited by 2,938
  6. 2017Arbitrary Style Transfer in Real-time with Adaptive Instance Normalizationcited by 1,998
  7. 2017Decoupled Weight Decay Regularizationcited by 1,722
  8. 2016WaveNet: A Generative Model for Raw Audiocited by 1,710
  9. 2018Spectral Normalization for Generative Adversarial Networkscited by 1,618
  10. 2021Alias-Free Generative Adversarial Networkscited by 1,181
  11. 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,139
  12. 2019StarGAN v2: Diverse Image Synthesis for Multiple Domainscited by 1,095
38 more
  1. 2017Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictionscited by 800
  2. 2018Multimodal Unsupervised Image-to-Image Translationcited by 795
  3. 2020FastSpeech 2: Fast and High-Quality End-to-End Text to Speechcited by 710
  4. 2019LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speechcited by 639
  5. 2019MaskGAN: Towards Diverse and Interactive Facial Image Manipulationcited by 624
  6. 2021Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speechcited by 555
  7. 2018ESPnet: End-to-End Speech Processing Toolkitcited by 554
  8. 2016Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networkscited by 541
  9. 2017Tacotron: Towards End-to-End Speech Synthesiscited by 487
  10. 2021Grad-TTS: A Diffusion Probabilistic Model for Text-to-Speechcited by 445
  11. 2021YourTTS: Towards Zero-Shot Multi-Speaker TTS and Zero-Shot Voice Conversion for everyonecited by 277
  12. 2020Glow-TTS: A Generative Flow for Text-to-Speech via Monotonic Alignment Searchcited by 269
  13. 2018Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesiscited by 257
  14. 2019SEAN: Image Synthesis with Semantic Region-Adaptive Normalizationcited by 253
  15. 2019Few-Shot Unsupervised Image-to-Image Translationcited by 222
  16. 2021A Survey on Neural Speech Synthesiscited by 182
  17. 2018Towards End-to-End Prosody Transfer for Expressive Speech Synthesis with Tacotroncited by 171
  18. 2022NaturalSpeech: End-to-End Text to Speech Synthesis with Human-Level Qualitycited by 151
  19. 2020FastPitch: Parallel Text-to-speech with Pitch Predictioncited by 150
  20. 2020Seen and Unseen emotional style transfer for voice conversion with a new emotional speech datasetcited by 119
  21. 2021ArtFlow: Unbiased Image Style Transfer via Reversible Neural Flowscited by 113
  22. 2021AdaSpeech: Adaptive Text to Speech for Custom Voicecited by 98
  23. 2018Hierarchical Generative Modeling for Controllable Speech Synthesiscited by 93
  24. 2021Meta-StyleSpeech : Multi-Speaker Adaptive Text-to-Speech Generationcited by 93
  25. 2021TorchAudio: Building Blocks for Audio and Speech Processingcited by 83
  26. 2019One-shot Voice Conversion by Separating Speaker and Content Representations with Instance Normalizationcited by 82
  27. 2018Learning latent representations for style control and transfer in end-to-end speech synthesiscited by 61
  28. 2021StarGANv2-VC: A Diverse, Unsupervised, Non-parallel Framework for Natural-Sounding Voice Conversioncited by 57
  29. 2020Flowtron: an Autoregressive Flow-based Generative Network for Text-to-Speech Synthesiscited by 54
  30. 2021PnG BERT: Augmented BERT on Phonemes and Graphemes for Neural TTScited by 52
  31. 2021PortaSpeech: Portable and High-Quality Generative Text-to-Speechcited by 47
  32. 2019Mellotron: Multispeaker expressive voice synthesis by conditioning on rhythm, pitch and global style tokenscited by 44
  33. 2020AGAIN-VC: A One-shot Voice Conversion using Activation Guidance and Adaptive Instance Normalizationcited by 44
  34. 2020MultiSpeech: Multi-Speaker Text to Speech with Transformercited by 42
  35. 2020Fully-hierarchical fine-grained prosody modeling for interpretable speech synthesiscited by 41
  36. 2021Parallel Tacotron 2: A Non-Autoregressive Neural TTS Model with Differentiable Duration Modelingcited by 31
  37. 2020Expressive TTS Training with Frame and Style Reconstruction Losscited by 30
  38. 2022JETS: Jointly Training FastSpeech2 and HiFi-GAN for End to End Text to Speechcited by 26

50 most cited of 54 All 54 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 7; 2024: 9; 2025: 11; 2026: 5.

32 papers.

Sort:

By area:

  1. 2023StyleTTS 2: Towards Human-Level Text-to-Speech through Style Diffusion and Adversarial Training with Large Speech Language Modelscited by 148
  2. 2025UniAVGen: Unified Audio and Video Generation with Asymmetric Cross-Modal Interactionscited by 35
  3. 2023Phoneme-Level BERT for Enhanced Prosody of Text-to-Speech with Grapheme Predictionscited by 22
  4. 2023The Ethical Implications of Generative Audio Models: A Systematic Literature Reviewcited by 18
  5. 2023HiFTNet: A Fast High-Quality Neural Vocoder with Harmonic-plus-Noise Filter and Inverse Short Time Fourier Transformcited by 10
  6. 2025Prosody-Enhanced Acoustic Pre-training and Acoustic-Disentangled Prosody Adapting for Movie Dubbingcited by 9
  7. 2024Controlling Emotion in Text-to-Speech with Natural Language Promptscited by 8
  8. 2024Style-Talker: Finetuning Audio Language Model and Style-Based Text-to-Speech Model for Fast Spoken Dialogue Generationcited by 7
  9. 2025DMOSpeech 2: Reinforcement Learning for Duration Prediction in Metric-Optimized Speech Synthesiscited by 7
  10. 2024StyleTTS-ZS: Efficient High-Quality Zero-Shot Text-to-Speech Synthesis with Distilled Time-Varying Style Diffusioncited by 6
  11. 2024DEX-TTS: Diffusion-based EXpressive Text-to-Speech with Style Modeling on Time Variabilitycited by 5
  12. 2024DMOSpeech: Direct Metric Optimization via Distilled Diffusion Model in Zero-Shot Speech Synthesiscited by 5
20 more
  1. 2025TCSinger 2: Customizable Multilingual Zero-shot Singing Voice Synthesiscited by 5
  2. 2024SF-Speech: Straightened Flow for Zero-Shot Voice Clonecited by 4
  3. 2025Voice Cloning: Comprehensive Surveycited by 4
  4. 2023SLMGAN: Exploiting Speech Language Model Representations for Unsupervised Zero-Shot Voice Conversion in GANscited by 3
  5. 2023Evaluating Speech Synthesis by Training Recognizers on Synthetic Speechcited by 3
  6. 2024Towards Evaluating the Robustness of Automatic Speech Recognition Systems via Audio Style Transfercited by 2
  7. 2025Continuous Autoregressive Modeling with Stochastic Monotonic Alignment for Speech Synthesiscited by 2
  8. 2023DurIAN-E: Duration Informed Attention Network For Expressive Text-to-Speech Synthesiscited by 1
  9. 2024Investigating Disentanglement in a Phoneme-level Speech Codec for Prosody Modelingcited by 1
  10. 2025RapFlow-TTS: Rapid and High-Fidelity Text-to-Speech with Improved Consistency Flow Matchingcited by 1
  11. 2025HiStyle: Hierarchical Style Embedding Predictor for Text-Prompt-Guided Controllable Speech Synthesiscited by 1
  12. 2024GTR-Voice: Articulatory Phonetics Informed Controllable Expressive Speech Synthesiscited by 0
  13. 2025Speaking images. A novel framework for the automated self-description of artworkscited by 0
  14. 2025LAPS-Diff: A Diffusion-Based Framework for Singing Voice Synthesis With Language Aware Prosody-Style Guided Learningcited by 0
  15. 2025FNH-TTS: Mixture-of-Experts Duration Modeling for Robust Neural Speech Synthesiscited by 0
  16. 2026DAST: A Dual-Stream Voice Anonymization Attacker with Staged Trainingcited by 0
  17. 2026Evaluation of Audio Language Models for Fairness, Safety, and Securitycited by 0
  18. 2026Multimodal Large Language Model-Enabled Video Translation: A Role-Oriented Surveycited by 0
  19. 2026Adaptive Oscillatory Inductive Bias for Modeling Sharp Prosodic Dynamics in Diffusion-Based TTScited by 0
  20. 2026Continuous-Time Acoustic Modelling with Neural Controlled Differential Equationscited by 0