Data to 7 Oct 2026

Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesis

arXiv 1803.09017 ↗First posted 23 Mar 2018cs.CL cs.LG cs.SDcited by 257

Not on the map. Why?

Abstract (arXiv)

In this work, we propose "global style tokens" (GSTs), a bank of embeddings that are jointly trained within Tacotron, a state-of-the-art end-to-end speech synthesis system. The embeddings are trained with no explicit labels, yet learn to model a large range of acoustic expressiveness. GSTs lead to a rich set of significant results. The soft interpretable "labels" they generate can be used to control synthesis in novel ways, such as varying speed and speaking style - independently of the text content. They can also be used for style transfer, replicating the speaking style of a single audio clip across an entire long-form text corpus. When trained on noisy, unlabeled found data, GSTs learn to factorize noise and speaker identity, providing a path towards highly scalable but robust speech synthesis.

Key facts. Cited by 257 papers in the map (data through 7 Oct 2026). First posted on arXiv 23 Mar 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 59; 2022: 55; 2023: 58; 2024: 33; 2025: 29; 2026: 23.

257 papers.

Sort:

By area:

2024NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Modelscited by 466
2023Voicebox: Text-Guided Multilingual Universal Speech Generation at Scalecited by 373
2023NaturalSpeech 2: Latent Diffusion Models are Natural and Zero-Shot Speech and Singing Synthesizerscited by 233
2021A Survey on Neural Speech Synthesiscited by 182
2023Noise2Music: Text-conditioned Music Generation with Diffusion Modelscited by 173
2022NaturalSpeech: End-to-End Text to Speech Synthesis with Human-Level Qualitycited by 151
2022PromptTTS: Controllable Text-to-Speech with Text Descriptionscited by 106
2023Guided Flows for Generative Modeling and Decision Makingcited by 99
2022ADD 2022: the First Audio Deep Synthesis Detection Challengecited by 94
2021Meta-StyleSpeech : Multi-Speaker Adaptive Text-to-Speech Generationcited by 93
2022ZeroEGGS: Zero-shot Example-based Gesture Generation from Speechcited by 83
2022FreeVC: Towards High-Quality Text-Free One-Shot Voice Conversioncited by 79

38 more

2023A Comprehensive Review of Data-Driven Co-Speech Gesture Generationcited by 66
2024Natural language guidance of high-fidelity text-to-speech with synthetic annotationscited by 66
2021PnG BERT: Augmented BERT on Phonemes and Graphemes for Neural TTScited by 52
2023Mega-TTS: Zero-Shot Text-to-Speech at Scale with Intrinsic Inductive Biascited by 51
2023PromptTTS 2: Describing and Generating Voices with Text Promptcited by 47
2021Half-Truth: A Partially Fake Audio Detection Datasetcited by 44
2022AdaSpeech 4: Adaptive Text to Speech in Zero-Shot Scenarioscited by 41
2022A3T: Alignment-Aware Acoustic and Text Pretraining for Speech Synthesis and Editingcited by 40
2024The VoicePrivacy 2024 Challenge Evaluation Plancited by 39
2021Investigating on Incorporating Pretrained and Learnable Speaker Representations for Multi-Speaker Multi-Style Text-to-Speechcited by 37
2022Emotion Intensity and its Control for Emotional Voice Conversioncited by 35
2023HierSpeech++: Bridging the Gap between Semantic and Acoustic Representation of Speech by Hierarchical Variational Inference for Zero-shot Speech Synthesiscited by 35
2022StyleTTS: A Style-Based Generative Model for Natural and Diverse Text-to-Speech Synthesiscited by 32
2022An Overview of Affective Speech Synthesis and Conversion in the Deep Learning Eracited by 32
2023E3 TTS: Easy End-to-End Diffusion-based Text to Speechcited by 30
2022Speech Synthesis with Mixed Emotionscited by 28
2023PromptTTS++: Controlling Speaker Identity in Prompt-Based Text-to-Speech Using Natural Language Descriptionscited by 25
2021fairseq S^2: A Scalable and Integrable Speech Synthesis Toolkitcited by 24
2021Synt++: Utilizing Imperfect Synthetic Data to Improve Speech Recognitioncited by 24
2021V2C: Visual Voice Cloningcited by 23
2021ESPnet2-TTS: Extending the Edge of TTS Researchcited by 22
2022ProsoSpeech: Enhancing Prosody With Quantized Vector Pre-training in Text-to-Speechcited by 22
2023A Vector Quantized Approach for Text to Speech Synthesis on Real-World Spontaneous Speechcited by 22
2024Towards Controllable Speech Synthesis in the Era of Large Language Models: A Systematic Surveycited by 22
2022GenerSpeech: Towards Style Transfer for Generalizable Out-Of-Domain Text-to-Speechcited by 21
2021Reinforcement Learning for Emotional Text-to-Speech Synthesis with Improved Emotion Discriminabilitycited by 19
2021DelightfulTTS: The Microsoft Speech Synthesis System for Blizzard Challenge 2021cited by 19
2023EmoMix: Emotion Mixing via Diffusion Models for Emotional Speech Synthesiscited by 19
2024EmoSphere++: Emotion-Controllable Zero-Shot Text-to-Speech via Emotion-Adaptive Spherical Vectorcited by 19
2021STYLER: Style Factor Modeling with Rapidity and Robustness via Speech Decomposition for Expressive and Controllable Neural Text to Speechcited by 18
2021Towards Multi-Scale Style Control for Expressive Speech Synthesiscited by 18
2021EMOVIE: A Mandarin Emotion Speech Dataset with a Simple Emotional Text-to-Speech Modelcited by 18
2023PromptVC: Flexible Stylistic Voice Conversion in Latent Space Driven by Natural Language Promptscited by 18
2023Emotion Rendering for Conversational Speech Synthesis with Heterogeneous Graph-Based Context Modelingcited by 17
2021Enhancing Speaking Styles in Conversational Text-to-Speech Synthesis with Graph-based Multi-modal Context Modelingcited by 15
2021Fine-grained style control in Transformer-based Text-to-speech Synthesiscited by 15
2022CampNet: Context-Aware Mask Prediction for End-to-End Text-Based Speech Editingcited by 15
2022Low-Resource Multilingual and Zero-Shot Multispeaker TTScited by 15

50 most cited of 257 All 257 (JSON)