mlsanityData to 7 Oct 2026

Meta-StyleSpeech : Multi-Speaker Adaptive Text-to-Speech Generation

arXiv 2106.03153 ↗First posted 6 Jun 2021eess.AS cs.CL cs.LGcited by 93

Not on the map. Why?

Cited by 20 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

With rapid progress in neural text-to-speech (TTS) models, personalized speech generation is now in high demand for many applications. For practical applicability, a TTS model should generate high-quality speech with only a few audio samples from the given speaker, that are also short in length. However, existing methods either require to fine-tune the model or achieve low adaptation quality without fine-tuning. In this work, we propose StyleSpeech, a new TTS model which not only synthesizes high-quality speech but also effectively adapts to new speakers. Specifically, we propose Style-Adaptive Layer Normalization (SALN) which aligns gain and bias of the text input according to the style extracted from a reference speech audio. With SALN, our model effectively synthesizes speech in the style of the target speaker even from single speech audio. Furthermore, to enhance StyleSpeech's adaptation to speech from new speakers, we extend it to Meta-StyleSpeech by introducing two discriminators trained with style prototypes, and performing episodic training. The experimental results show that our models generate high-quality speech which accurately follows the speaker's voice with single short-duration (1-3 sec) speech audio, significantly outperforming baselines.

Key facts. Cited by 93 papers in the map (data through 7 Oct 2026). Builds on 37 of its 50 references that are in the map. First posted on arXiv 6 Jun 2021.

Builds on

37 of 50 references.

References · 37

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  3. 2016Layer Normalizationcited by 5,579
  4. 2017Prototypical Networks for Few-shot Learningcited by 3,001
  5. 2016Matching Networks for One Shot Learningcited by 2,246
  6. 2016WaveNet: A Generative Model for Raw Audiocited by 1,710
  7. 2018Spectral Normalization for Generative Adversarial Networkscited by 1,618
  8. 2016Least Squares Generative Adversarial Networkscited by 1,000
  9. 2017Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictionscited by 800
  10. 2020FastSpeech 2: Fast and High-Quality End-to-End Text to Speechcited by 710
  11. 2016Language Modeling with Gated Convolutional Networkscited by 688
  12. 2019LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speechcited by 639
25 more
  1. 2017Tacotron: Towards End-to-End Speech Synthesiscited by 487
  2. 2015Deep Speech 2: End-to-End Speech Recognition in English and Mandarincited by 449
  3. 2018TADAM: Task dependent adaptive metric for improved few-shot learningcited by 408
  4. 2019MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesiscited by 351
  5. 2017Generalized End-to-End Loss for Speaker Verificationcited by 290
  6. 2018Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesiscited by 280
  7. 2018Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesiscited by 257
  8. 2018cGANs with Projection Discriminatorcited by 253
  9. 2018Towards End-to-End Prosody Transfer for Expressive Speech Synthesis with Tacotroncited by 171
  10. 2020FastPitch: Parallel Text-to-speech with Pitch Predictioncited by 150
  11. 2019Mish: A Self Regularized Non-Monotonic Activation Functioncited by 141
  12. 2018Neural Voice Cloning with a Few Samplescited by 100
  13. 2021AdaSpeech: Adaptive Text to Speech for Custom Voicecited by 98
  14. 2018Hierarchical Generative Modeling for Controllable Speech Synthesiscited by 93
  15. 2017Deep Voice: Real-time Neural Text-to-Speechcited by 83
  16. 2017Deep Voice 2: Multi-Speaker Neural Text-to-Speechcited by 75
  17. 2017Deep Voice 3: Scaling Text-to-Speech with Convolutional Sequence Learningcited by 67
  18. 2019FIGR: Few-shot Image Generation with Reptilecited by 54
  19. 2018Sample Efficient Adaptive Text-to-Speechcited by 45
  20. 2016One-Shot Generalization in Deep Generative Modelscited by 44
  21. 2020MultiSpeech: Multi-Speaker Text to Speech with Transformercited by 42
  22. 2017Few-shot Autoregressive Density Estimation: Towards Learning to Learn Distributionscited by 24
  23. 2020BOFFIN TTS: Few-Shot Speaker Adaptation by Bayesian Optimizationcited by 22
  24. 2018Fitting New Speakers Based on a Short Untranscribed Samplecited by 18
  25. 2020AdaDurIAN: Few-shot Adaptation for Neural Text-to-Speech with DurIANcited by 14

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 3; 2022: 16; 2023: 25; 2024: 22; 2025: 19; 2026: 8.

93 papers.

Sort:

By area:

  1. 2022Multimodal Learning with Transformers: A Surveycited by 399
  2. 2021A Survey on Neural Speech Synthesiscited by 182
  3. 2022ProDiff: Progressive Fast Diffusion Model For High-Quality Text-to-Speechcited by 147
  4. 2024Lumina-T2X: Transforming Text into Any Modality, Resolution, and Duration via Flow-based Large Diffusion Transformerscited by 137
  5. 2024EMOVA: Empowering Language Models to See, Hear and Speak with Vivid Emotionscited by 44
  6. 2022AdaSpeech 4: Adaptive Text to Speech in Zero-Shot Scenarioscited by 41
  7. 2023DDDM-VC: Decoupled Denoising Diffusion Models with Disentangled Representation and Prior Mixup for Verified Robust Voice Conversioncited by 38
  8. 2022Guided-TTS 2: A Diffusion Model for High-quality Adaptive Text-to-Speech with Untranscribed Datacited by 35
  9. 2023HierSpeech++: Bridging the Gap between Semantic and Acoustic Representation of Speech by Hierarchical Variational Inference for Zero-shot Speech Synthesiscited by 35
  10. 2022Grad-StyleSpeech: Any-speaker Adaptive Text-to-Speech Synthesis with Diffusion Modelscited by 34
  11. 2022StyleTTS: A Style-Based Generative Model for Natural and Diverse Text-to-Speech Synthesiscited by 32
  12. 2023Diff-HierVC: Diffusion-based Hierarchical Voice Conversion with Robust Pitch Generation and Masked Prior for Zero-shot Speaker Adaptationcited by 30
38 more
  1. 2023Make-A-Voice: Unified Voice Synthesis With Discrete Representationcited by 27
  2. 2023FluentSpeech: Stutter-Oriented Automatic Speech Editing with Context-Aware Diffusion Modelscited by 23
  3. 2024Towards Controllable Speech Synthesis in the Era of Large Language Models: A Systematic Surveycited by 22
  4. 2022GenerSpeech: Towards Style Transfer for Generalizable Out-Of-Domain Text-to-Speechcited by 21
  5. 2023StyleSinger: Style Transfer for Out-of-Domain Singing Voice Synthesiscited by 21
  6. 2024Faces that Speak: Jointly Synthesising Talking Face and Speech from Textcited by 21
  7. 2024StyleDubber: Towards Multi-Scale Style Learning for Movie Dubbingcited by 20
  8. 2023The Ethical Implications of Generative Audio Models: A Systematic Literature Reviewcited by 18
  9. 2023PromptVC: Flexible Stylistic Voice Conversion in Latent Space Driven by Natural Language Promptscited by 18
  10. 2022SpeechPainter: Text-conditioned Speech Inpaintingcited by 17
  11. 2023Mimic: Speaking Style Disentanglement for Speech-Driven 3D Facial Animationcited by 16
  12. 2023DiffProsody: Diffusion-based Latent Prosody Generation for Expressive Speech Synthesis with Prosody Conditional Adversarial Trainingcited by 15
  13. 2023ZET-Speech: Zero-shot adaptive Emotion-controllable Text-to-Speech Synthesis with Diffusion and Style-based Modelscited by 14
  14. 2022Heterogeneous Target Speech Separationcited by 13
  15. 2023DiffVoice: Text-to-Speech with Latent Diffusioncited by 13
  16. 2022Transfer Learning Framework for Low-Resource Text-to-Speech using a Large-Scale Unlabeled Speech Corpuscited by 10
  17. 2023A Holistic Cascade System, benchmark, and Human Evaluation Protocol for Expressive Speech-to-Speech Translationcited by 10
  18. 2023HierVST: Hierarchical Adaptive Zero-shot Voice Style Transfercited by 9
  19. 2025Prosody-Enhanced Acoustic Pre-training and Acoustic-Disentangled Prosody Adapting for Movie Dubbingcited by 9
  20. 2022NoreSpeech: Knowledge Distillation based Conditional Diffusion Model for Noise-robust Expressive TTScited by 8
  21. 2023HM-Conformer: A Conformer-based audio deepfake detection system with hierarchical pooling and multi-level classification token aggregation methodscited by 8
  22. 2024DurFlex-EVC: Duration-Flexible Emotional Voice Conversion Leveraging Discrete Representations without Text Alignmentcited by 8
  23. 2024RapVerse: Coherent Vocals and Whole-Body Motions Generations from Textcited by 7
  24. 2025DMOSpeech 2: Reinforcement Learning for Duration Prediction in Metric-Optimized Speech Synthesiscited by 7
  25. 2026PersonaGesture: Single-Reference Co-Speech Gesture Personalization for Unseen Speakerscited by 7
  26. 2022Text-driven Emotional Style Control and Cross-speaker Style Transfer in Neural TTScited by 6
  27. 2023M2-CTTS: End-to-End Multi-scale Multi-modal Conversational Text-to-Speech Synthesiscited by 6
  28. 2024USAT: A Universal Speaker-Adaptive Text-to-Speech Approachcited by 6
  29. 2024Style Mixture of Experts for Expressive Text-To-Speech Synthesiscited by 6
  30. 2024StyleTTS-ZS: Efficient High-Quality Zero-Shot Text-to-Speech Synthesis with Distilled Time-Varying Style Diffusioncited by 6
  31. 2021Meta-Voice: Fast few-shot style transfer for expressive voice cloning using meta learningcited by 5
  32. 2023Generalizable Zero-Shot Speaker Adaptive Speech Synthesis with Disentangled Representationscited by 5
  33. 2023Self-Supervised Disentangled Representation Learning for Robust Target Speech Extractioncited by 5
  34. 2024DEX-TTS: Diffusion-based EXpressive Text-to-Speech with Style Modeling on Time Variabilitycited by 5
  35. 2024DMOSpeech: Direct Metric Optimization via Distilled Diffusion Model in Zero-Shot Speech Synthesiscited by 5
  36. 2025DiEmo-TTS: Disentangled Emotion Representations via Self-Supervised Distillation for Cross-Speaker Emotion Transfer in Text-to-Speechcited by 5
  37. 2024SF-Speech: Straightened Flow for Zero-Shot Voice Clonecited by 4
  38. 2025FaceSpeak: Expressive and High-Quality Speech Synthesis from Human Portraits of Different Stylescited by 4

50 most cited of 93 All 93 (JSON)