# Adversarially Trained Multi-Singer Sequence-To-Sequence Singing Synthesizer

arXiv: https://arxiv.org/abs/2006.10317 · HTML: https://mlsanity.com/paper/2006.10317/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Jun 2020.

## Built on by (most cited first, 12 of 12)

- [DiffSinger: Singing Voice Synthesis via Shallow Diffusion Mechanism](https://mlsanity.com/paper/2105.02446/) (2021)
- [Learning the Beauty in Songs: Neural Singing Voice Beautifier](https://mlsanity.com/paper/2202.13277/) (2022)
- [RefineGAN: Universally Generating Waveform Better than Ground Truth with Highly Accurate Pitch and Intensity Responses](https://mlsanity.com/paper/2111.00962/) (2021)
- [PauseSpeech: Natural Speech Synthesis via Pre-trained Language Model and Pause-based Prosody Modeling](https://mlsanity.com/paper/2306.07489/) (2023)
- [SingAug: Data Augmentation for Singing Voice Synthesis with Cycle-consistent Training Strategy](https://mlsanity.com/paper/2203.17001/) (2022)
- [AlignSTS: Speech-to-Singing Conversion via Cross-Modal Alignment](https://mlsanity.com/paper/2305.04476/) (2023)
- [RDSinger: Reference-based Diffusion Network for Singing Voice Synthesis](https://mlsanity.com/paper/2410.21641/) (2024)
- [Karaoker: Alignment-free singing voice synthesis with speech training data](https://mlsanity.com/paper/2204.04127/) (2022)
- [SmoothSinger: A Conditional Diffusion Model for Singing Voice Synthesis with Multi-Resolution Architecture](https://mlsanity.com/paper/2506.21478/) (2025)
- [Pitch Preservation In Singing Voice Synthesis](https://mlsanity.com/paper/2110.05033/) (2021)
- [Singing Voice Synthesis Based on a Musical Note Position-Aware Attention Mechanism](https://mlsanity.com/paper/2212.13703/) (2022)
- [Singing voice synthesis based on frame-level sequence-to-sequence models considering vocal timing deviation](https://mlsanity.com/paper/2301.02262/) (2023)

## Abstract (arXiv)

This paper presents a high quality singing synthesizer that is able to model a voice with limited available recordings. Based on the sequence-to-sequence singing model, we design a multi-singer framework to leverage all the existing singing data of different singers. To attenuate the issue of musical score unbalance among singers, we incorporate an adversarial task of singer classification to make encoder output less singer dependent. Furthermore, we apply multiple random window discriminators (MRWDs) on the generated acoustic features to make the network be a GAN. Both objective and subjective evaluations indicate that the proposed synthesizer can generate higher quality singing voice than baseline (4.12 vs 3.53 in MOS). Especially, the articulation of high-pitched vowels is significantly enhanced.