# TCSinger: Zero-Shot Singing Voice Synthesis with Style Transfer and Multi-Level Style Control

arXiv: https://arxiv.org/abs/2409.15977 · HTML: https://mlsanity.com/paper/2409.15977/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 39 references that are in the map.
- First posted on arXiv 24 Sep 2024.

## Builds on (most cited first, 30 of 30)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://mlsanity.com/paper/2106.07447/) (2021)
- [WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing](https://mlsanity.com/paper/2110.13900/) (2021)
- [Generating Diverse High-Fidelity Images with VQ-VAE-2](https://mlsanity.com/paper/1906.00446/) (2019)
- [HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis](https://mlsanity.com/paper/2010.05646/) (2020)
- [FastSpeech 2: Fast and High-Quality End-to-End Text to Speech](https://mlsanity.com/paper/2006.04558/) (2020)
- [Vector-quantized Image Modeling with Improved VQGAN](https://mlsanity.com/paper/2110.04627/) (2021)
- [YourTTS: Towards Zero-Shot Multi-Speaker TTS and Zero-Shot Voice Conversion for everyone](https://mlsanity.com/paper/2112.02418/) (2021)
- [DiffSinger: Singing Voice Synthesis via Shallow Diffusion Mechanism](https://mlsanity.com/paper/2105.02446/) (2021)
- [ProDiff: Progressive Fast Diffusion Model For High-Quality Text-to-Speech](https://mlsanity.com/paper/2207.06389/) (2022)
- [MoVQ: Modulating Quantized Vectors for High-Fidelity Image Generation](https://mlsanity.com/paper/2209.09002/) (2022)
- [AISHELL-3: A Multi-speaker Mandarin TTS Corpus and the Baselines](https://mlsanity.com/paper/2010.11567/) (2020)
- [Zero-Shot Multi-Speaker Text-To-Speech with State-of-the-art Neural Speaker Embeddings](https://mlsanity.com/paper/1910.10838/) (2019)
- [Multi-Singer: Fast Multi-Singer Singing Voice Vocoder With A Large-Scale Corpus](https://mlsanity.com/paper/2112.10358/) (2021)
- [Mega-TTS: Zero-Shot Text-to-Speech at Scale with Intrinsic Inductive Bias](https://mlsanity.com/paper/2306.03509/) (2023)
- [GTSinger: A Global Multi-Technique Singing Corpus with Realistic Music Scores for All Singing Tasks](https://mlsanity.com/paper/2409.13832/) (2024)
- [StyleSinger: Style Transfer for Out-of-Domain Singing Voice Synthesis](https://mlsanity.com/paper/2312.10741/) (2023)
- [STYLER: Style Factor Modeling with Rapidity and Robustness via Speech Decomposition for Expressive and Controllable Neural Text to Speech](https://mlsanity.com/paper/2103.09474/) (2021)
- [RMSSinger: Realistic-Music-Score based Singing Voice Synthesis](https://mlsanity.com/paper/2305.10686/) (2023)
- [Prompt-Singer: Controllable Singing-Voice-Synthesis with Natural Language Prompt](https://mlsanity.com/paper/2403.11780/) (2024)
- [Learning the Beauty in Songs: Neural Singing Voice Beautifier](https://mlsanity.com/paper/2202.13277/) (2022)
- [Muskits: an End-to-End Music Processing Toolkit for Singing Voice Synthesis](https://mlsanity.com/paper/2205.04029/) (2022)
- [Robust Singing Voice Transcription Serves Synthesis](https://mlsanity.com/paper/2405.09940/) (2024)
- [A Melody-Unsupervision Model for Singing Voice Synthesis](https://mlsanity.com/paper/2110.06546/) (2021)
- [MuSE-SVS: Multi-Singer Emotional Singing Voice Synthesizer that Controls Emotional Intensity](https://mlsanity.com/paper/2203.00931/) (2022)
- [Adversarial Multi-Task Learning for Disentangling Timbre and Pitch in Singing Voice Synthesis](https://mlsanity.com/paper/2206.11558/) (2022)
- [WeSinger 2: Fully Parallel Singing Voice Synthesis via Multi-Singer Conditional Adversarial Training](https://mlsanity.com/paper/2207.01886/) (2022)

## Built on by (most cited first, 20 of 20)

- [SingMOS-Pro: An Comprehensive Benchmark for Singing Quality Assessment](https://mlsanity.com/paper/2510.01812/) (2025)
- [TCSinger 2: Customizable Multilingual Zero-shot Singing Voice Synthesis](https://mlsanity.com/paper/2505.14910/) (2025)
- [Vevo2: A Unified and Controllable Framework for Speech and Singing Voice Generation](https://mlsanity.com/paper/2508.16332/) (2025)
- [YingMusic-Singer: Zero-shot Singing Voice Synthesis and Editing with Annotation-free Melody Guidance](https://mlsanity.com/paper/2512.04779/) (2025)
- [Versatile Framework for Song Generation with Prompt-based Control](https://mlsanity.com/paper/2504.19062/) (2025)
- [Conan: A Chunkwise Online Network for Zero-Shot Adaptive Voice Conversion](https://mlsanity.com/paper/2507.14534/) (2025)
- [DiTSinger: Scaling Singing Voice Synthesis with Diffusion Transformer and Implicit Alignment](https://mlsanity.com/paper/2510.09016/) (2025)
- [SoulX-Singer: Towards High-Quality Zero-Shot Singing Voice Synthesis](https://mlsanity.com/paper/2602.07803/) (2026)
- [VibE-SVC: Vibrato Extraction with High-frequency F0 Contour for Singing Voice Conversion](https://mlsanity.com/paper/2505.20794/) (2025)
- [Spotlight-TTS: Spotlighting the Style via Voiced-Aware Style Extraction and Style Direction Adjustment for Expressive Text-to-Speech](https://mlsanity.com/paper/2505.20868/) (2025)
- [SmoothSinger: A Conditional Diffusion Model for Singing Voice Synthesis with Multi-Resolution Architecture](https://mlsanity.com/paper/2506.21478/) (2025)
- [Towards Streaming Synchronized Spatial Audio Generation via Autoregressive Diffusion Transformer](https://mlsanity.com/paper/2605.30940/) (2026)
- [Leveraging Pretrained Diffusion Models for Zero-Shot Part Assembly](https://mlsanity.com/paper/2505.00426/) (2025)
- [LAPS-Diff: A Diffusion-Based Framework for Singing Voice Synthesis With Language Aware Prosody-Style Guided Learning](https://mlsanity.com/paper/2507.04966/) (2025)
- [Voiced-Aware Style Extraction and Style Direction Adjustment for Expressive Text-to-Speech](https://mlsanity.com/paper/2511.14824/) (2025)
- [Tutti: Expressive Multi-Singer Synthesis via Structure-Level Timbre Control and Vocal Texture Modeling](https://mlsanity.com/paper/2602.08233/) (2026)
- [Controllable Singing Style Conversion with Boundary-Aware Information Bottleneck](https://mlsanity.com/paper/2604.05526/) (2026)
- [Vibrato Expression Control for Singing Voice Conversion with Improving Independent Control](https://mlsanity.com/paper/2606.17126/) (2026)
- [VocalRender: Score-Native Singing Voice Synthesis for Real-World Composition](https://mlsanity.com/paper/2607.27768/) (2026)
- [CookVoice: Unified Framework for Style Controllable Multi-Modal Human Voice Generation](https://mlsanity.com/paper/2608.11590/) (2026)

## Abstract (arXiv)

Zero-shot singing voice synthesis (SVS) with style transfer and style control aims to generate high-quality singing voices with unseen timbres and styles (including singing method, emotion, rhythm, technique, and pronunciation) from audio and text prompts. However, the multifaceted nature of singing styles poses a significant challenge for effective modeling, transfer, and control. Furthermore, current SVS models often fail to generate singing voices rich in stylistic nuances for unseen singers. To address these challenges, we introduce TCSinger, the first zero-shot SVS model for style transfer across cross-lingual speech and singing styles, along with multi-level style control. Specifically, TCSinger proposes three primary modules: 1) the clustering style encoder employs a clustering vector quantization model to stably condense style information into a compact latent space; 2) the Style and Duration Language Model (S&D-LM) concurrently predicts style information and phoneme duration, which benefits both; 3) the style adaptive decoder uses a novel mel-style adaptive normalization method to generate singing voices with enhanced details. Experimental results show that TCSinger outperforms all baseline models in synthesis quality, singer similarity, and style controllability across various tasks, including zero-shot style transfer, multi-level style control, cross-lingual style transfer, and speech-to-singing style transfer. Singing voice samples can be accessed at https://aaronz345.github.io/TCSingerDemo/.