# UniSyn: An End-to-End Unified Model for Text-to-Speech and Singing Voice Synthesis

arXiv: https://arxiv.org/abs/2212.01546 · HTML: https://mlsanity.com/paper/2212.01546/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Dec 2022.

## Built on by (most cited first, 4 of 4)

- [Vevo2: A Unified and Controllable Framework for Speech and Singing Voice Generation](https://mlsanity.com/paper/2508.16332/) (2025)
- [SoulX-Singer: Towards High-Quality Zero-Shot Singing Voice Synthesis](https://mlsanity.com/paper/2602.07803/) (2026)
- [LHQ-SVC: Lightweight and High Quality Singing Voice Conversion Modeling](https://mlsanity.com/paper/2409.08583/) (2024)
- [An End-to-End Approach for Chord-Conditioned Song Generation](https://mlsanity.com/paper/2409.06307/) (2024)

## Abstract (arXiv)

Text-to-speech (TTS) and singing voice synthesis (SVS) aim at generating high-quality speaking and singing voice according to textual input and music scores, respectively. Unifying TTS and SVS into a single system is crucial to the applications requiring both of them. Existing methods usually suffer from some limitations, which rely on either both singing and speaking data from the same person or cascaded models of multiple tasks. To address these problems, a simplified elegant framework for TTS and SVS, named UniSyn, is proposed in this paper. It is an end-to-end unified model that can make a voice speak and sing with only singing or speaking data from this person. To be specific, a multi-conditional variational autoencoder (MC-VAE), which constructs two independent latent sub-spaces with the speaker- and style-related (i.e. speak or sing) conditions for flexible control, is proposed in UniSyn. Moreover, supervised guided-VAE and timbre perturbation with the Wasserstein distance constraint are leveraged to further disentangle the speaker timbre and style. Experiments conducted on two speakers and two singers demonstrate that UniSyn can generate natural speaking and singing voice without corresponding training data. The proposed approach outperforms the state-of-the-art end-to-end voice generation work, which proves the effectiveness and advantages of UniSyn.