mlsanityData to 7 Oct 2026

Style-Label-Free: Cross-Speaker Style Transfer by Quantized VAE and Speaker-wise Normalization in Speech Synthesis

arXiv 2212.06397 ↗First posted 13 Dec 2022cs.SD cs.AI cs.CLcited by 7

Not on the map. Why?

Abstract (arXiv)

Cross-speaker style transfer in speech synthesis aims at transferring a style from source speaker to synthesised speech of a target speaker's timbre. Most previous approaches rely on data with style labels, but manually-annotated labels are expensive and not always reliable. In response to this problem, we propose Style-Label-Free, a cross-speaker style transfer method, which can realize the style transfer from source speaker to target speaker without style labels. Firstly, a reference encoder structure based on quantized variational autoencoder (Q-VAE) and style bottleneck is designed to extract discrete style representations. Secondly, a speaker-wise batch normalization layer is proposed to reduce the source speaker leakage. In order to improve the style extraction ability of the reference encoder, a style invariant and contrastive data augmentation method is proposed. Experimental results show that the method outperforms the baseline. We provide a website with audio samples.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 27 of its 36 references that are in the map. First posted on arXiv 13 Dec 2022.

Builds on

27 of 36 references.

References · 27

  1. 2017Squeeze-and-Excitation Networkscited by 4,763
  2. 2017Neural Discrete Representation Learningcited by 4,650
  3. 2013Auto-Encoding Variational Bayescited by 3,980
  4. 2015Domain-Adversarial Training of Neural Networkscited by 3,023
  5. 2017Tacotron: Towards End-to-End Speech Synthesiscited by 487
  6. 2020Glow-TTS: A Generative Flow for Text-to-Speech via Monotonic Alignment Searchcited by 269
  7. 2018Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesiscited by 257
  8. 2018Towards End-to-End Prosody Transfer for Expressive Speech Synthesis with Tacotroncited by 171
  9. 2020Seen and Unseen emotional style transfer for voice conversion with a new emotional speech datasetcited by 119
  10. 2018Hierarchical Generative Modeling for Controllable Speech Synthesiscited by 93
  11. 2018Learning latent representations for style control and transfer in end-to-end speech synthesiscited by 61
  12. 2020Parallel Tacotron: Non-Autoregressive and Controllable TTScited by 49
15 more
  1. 2020Generating diverse and natural text-to-speech samples using a quantized fine-grained VAE and auto-regressive prosody priorcited by 33
  2. 2020Controllable Emotion Transfer For End-to-End Speech Synthesiscited by 32
  3. 2019CHiVE: Varying Prosody in Speech Synthesis with a Linguistically Driven Dynamic Hierarchical Conditional Variational Networkcited by 25
  4. 2019Semi-Supervised Generative Modeling for Controllable Speech Synthesiscited by 21
  5. 2018Fitting New Speakers Based on a Short Untranscribed Samplecited by 18
  6. 2021STYLER: Style Factor Modeling with Rapidity and Robustness via Speech Decomposition for Expressive and Controllable Neural Text to Speechcited by 18
  7. 2020DenoiSpeech: Denoising Text to Speech with Frame-Level Noise Modelingcited by 13
  8. 2019Multi-Reference Neural TTS Stylization with Adversarial Cycle Consistencycited by 12
  9. 2021VARA-TTS: Non-Autoregressive Text-to-Speech Synthesis based on Very Deep VAE with Residual Attentioncited by 12
  10. 2019Multi-reference Tacotron by Intercross Training for Style Disentangling,Transfer and Control in Speech Synthesiscited by 11
  11. 2021Cross-speaker Style Transfer with Prosody Bottleneck in Neural Speech Synthesiscited by 9
  12. 2021Cross-speaker Emotion Transfer Based on Speaker Condition Layer Normalization and Semi-Supervised Training in Text-To-Speechcited by 8
  13. 2021Improving Performance of Seen and Unseen Speech Style Transfer in End-to-end Neural TTScited by 6
  14. 2021Multi-speaker Multi-style Text-to-speech Synthesis With Single-speaker Single-style Training Data Scenarioscited by 5
  15. 2021UniTTS: Residual Learning of Unified Embedding Space for Speech Style Controlcited by 1

Built on by

Cited by year

Citing papers per year: 2023: 4; 2024: 1; 2025: 2.

7 papers.

  1. 2025Kling-Foley: Multimodal Diffusion Transformer for High-Quality Video-to-Audio Generationcited by 38
  2. 2023Minimally-Supervised Speech Synthesis with Conditional Diffusion Model and Language Model: A Comparative Study of Semantic Codingcited by 10
  3. 2023Improving Prosody for Cross-Speaker Style Transfer by Semi-Supervised Style Extractor and Hierarchical Modeling in Speech Synthesiscited by 7
  4. 2024VQ-CTAP: Cross-Modal Fine-Grained Sequence Representation Learning for Speech Processingcited by 7
  5. 2023High-Fidelity Speech Synthesis with Minimal Supervision: All Using Diffusion Modelscited by 6
  6. 2025SecoustiCodec: Cross-Modal Aligned Streaming Single-Codecbook Speech Codeccited by 5
  7. 2023PITS: Variational Pitch Inference without Fundamental Frequency for End-to-End Pitch-controllable TTScited by 4