# GSA-TTS : Toward Zero-Shot Speech Synthesis based on Gradual Style Adaptor

arXiv: https://arxiv.org/abs/2505.19384 · HTML: https://mlsanity.com/paper/2505.19384/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 22 of its 30 references that are in the map.
- First posted on arXiv 26 May 2025.

## Builds on (most cited first, 22 of 22)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis](https://mlsanity.com/paper/2010.05646/) (2020)
- [Language Modeling with Gated Convolutional Networks](https://mlsanity.com/paper/1612.08083/) (2016)
- [Tacotron: Towards End-to-End Speech Synthesis](https://mlsanity.com/paper/1703.10135/) (2017)
- [Are Transformers universal approximators of sequence-to-sequence functions?](https://mlsanity.com/paper/1912.10077/) (2019)
- [Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesis](https://mlsanity.com/paper/1803.09017/) (2018)
- [FastPitch: Parallel Text-to-speech with Pitch Prediction](https://mlsanity.com/paper/2006.06873/) (2020)
- [LibriTTS-R: A Restored Multi-Speaker Text-to-Speech Corpus](https://mlsanity.com/paper/2305.18802/) (2023)
- [AdaSpeech: Adaptive Text to Speech for Custom Voice](https://mlsanity.com/paper/2103.00993/) (2021)
- [Meta-StyleSpeech : Multi-Speaker Adaptive Text-to-Speech Generation](https://mlsanity.com/paper/2106.03153/) (2021)
- [Zero-Shot Multi-Speaker Text-To-Speech with State-of-the-art Neural Speaker Embeddings](https://mlsanity.com/paper/1910.10838/) (2019)
- [One TTS Alignment To Rule Them All](https://mlsanity.com/paper/2108.10447/) (2021)
- [Sequence-to-Sequence Neural Net Models for Grapheme-to-Phoneme Conversion](https://mlsanity.com/paper/1506.00196/) (2015)
- [g2pM: A Neural Grapheme-to-Phoneme Conversion Package for Mandarin Chinese Based on a New Open Benchmark Dataset](https://mlsanity.com/paper/2004.03136/) (2020)
- [HierVST: Hierarchical Adaptive Zero-shot Voice Style Transfer](https://mlsanity.com/paper/2307.16171/) (2023)
- [CrossSpeech: Speaker-independent Acoustic Representation for Cross-lingual Speech Synthesis](https://mlsanity.com/paper/2302.14370/) (2023)
- [Visual Question Answering based on Local-Scene-Aware Referring Expression Generation](https://mlsanity.com/paper/2101.08978/) (2021)
- [Good Neighbors Are All You Need for Chinese Grapheme-to-Phoneme Conversion](https://mlsanity.com/paper/2303.07726/) (2023)
- [Intelli-Z: Toward Intelligible Zero-Shot TTS](https://mlsanity.com/paper/2401.13921/) (2024)

## Abstract (arXiv)

We present the gradual style adaptor TTS (GSA-TTS) with a novel style encoder that gradually encodes speaking styles from an acoustic reference for zero-shot speech synthesis. GSA first captures the local style of each semantic sound unit. Then the local styles are combined by self-attention to obtain a global style condition. This semantic and hierarchical encoding strategy provides a robust and rich style representation for an acoustic model. We test GSA-TTS on unseen speakers and obtain promising results regarding naturalness, speaker similarity, and intelligibility. Additionally, we explore the potential of GSA in terms of interpretability and controllability, which stems from its hierarchical structure.