# WeSinger: Data-augmented Singing Voice Synthesis with Auxiliary Losses

arXiv: https://arxiv.org/abs/2203.10750 · HTML: https://mlsanity.com/paper/2203.10750/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 27 references that are in the map.
- First posted on arXiv 21 Mar 2022.

## Builds on (most cited first, 17 of 17)

- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Conformer: Convolution-augmented Transformer for Speech Recognition](https://mlsanity.com/paper/2005.08100/) (2020)
- [HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis](https://mlsanity.com/paper/2010.05646/) (2020)
- [Tacotron: Towards End-to-End Speech Synthesis](https://mlsanity.com/paper/1703.10135/) (2017)
- [Efficient Neural Audio Synthesis](https://mlsanity.com/paper/1802.08435/) (2018)
- [LPCNet: Improving Neural Speech Synthesis Through Linear Prediction](https://mlsanity.com/paper/1810.11846/) (2018)
- [Learning to Speak Fluently in a Foreign Language: Multilingual Speech Synthesis and Cross-Language Voice Cloning](https://mlsanity.com/paper/1907.04448/) (2019)
- [Multi-Singer: Fast Multi-Singer Singing Voice Vocoder With A Large-Scale Corpus](https://mlsanity.com/paper/2112.10358/) (2021)
- [Opencpop: A High-Quality Open Source Chinese Popular Song Corpus for Singing Voice Synthesis](https://mlsanity.com/paper/2201.07429/) (2022)
- [HiFiSinger: Towards High-Fidelity Neural Singing Voice Synthesis](https://mlsanity.com/paper/2009.01776/) (2020)
- [XiaoiceSing: A High-Quality and Integrated Singing Voice Synthesis System](https://mlsanity.com/paper/2006.06261/) (2020)
- [DeepSinger: Singing Voice Synthesis with Data Mined From the Web](https://mlsanity.com/paper/2007.04590/) (2020)
- [ByteSing: A Chinese Singing Voice Synthesis System Using Duration Allocated Encoder-Decoder Acoustic Models and WaveRNN Vocoders](https://mlsanity.com/paper/2004.11012/) (2020)
- [A Neural Parametric Singing Synthesizer](https://mlsanity.com/paper/1704.03809/) (2017)
- [Sinsy: A Deep Neural Network-Based Singing Voice Synthesis System](https://mlsanity.com/paper/2108.02776/) (2021)
- [Singing voice synthesis based on convolutional neural networks](https://mlsanity.com/paper/1904.06868/) (2019)
- [N-Singer: A Non-Autoregressive Korean Singing Voice Synthesis System for Pronunciation Enhancement](https://mlsanity.com/paper/2106.15205/) (2021)

## Built on by (most cited first, 7 of 7)

- [GTSinger: A Global Multi-Technique Singing Corpus with Realistic Music Scores for All Singing Tasks](https://mlsanity.com/paper/2409.13832/) (2024)
- [RMSSinger: Realistic-Music-Score based Singing Voice Synthesis](https://mlsanity.com/paper/2305.10686/) (2023)
- [Prompt-Singer: Controllable Singing-Voice-Synthesis with Natural Language Prompt](https://mlsanity.com/paper/2403.11780/) (2024)
- [RapVerse: Coherent Vocals and Whole-Body Motions Generations from Text](https://mlsanity.com/paper/2405.20336/) (2024)
- [YingMusic-Singer: Zero-shot Singing Voice Synthesis and Editing with Annotation-free Melody Guidance](https://mlsanity.com/paper/2512.04779/) (2025)
- [SingAug: Data Augmentation for Singing Voice Synthesis with Cycle-consistent Training Strategy](https://mlsanity.com/paper/2203.17001/) (2022)
- [Enhancing the vocal range of single-speaker singing voice synthesis with melody-unsupervised pre-training](https://mlsanity.com/paper/2309.00284/) (2023)

## Abstract (arXiv)

In this paper, we develop a new multi-singer Chinese neural singing voice synthesis (SVS) system named WeSinger. To improve the accuracy and naturalness of synthesized singing voice, we design several specifical modules and techniques: 1) A deep bi-directional LSTM-based duration model with multi-scale rhythm loss and post-processing step; 2) A Transformer-alike acoustic model with progressive pitch-weighted decoder loss; 3) a 24 kHz pitch-aware LPCNet neural vocoder to produce high-quality singing waveforms; 4) A novel data augmentation method with multi-singer pre-training for stronger robustness and naturalness. To our knowledge, WeSinger is the first SVS system to adopt 24 kHz LPCNet and multi-singer pre-training simultaneously. Both quantitative and qualitative evaluation results demonstrate the effectiveness of WeSinger in terms of accuracy and naturalness, and WeSinger achieves state-of-the-art performance on the recent public Chinese singing corpus Opencpopfootnotehttps://wenet.org.cn/opencpop/. Some synthesized singing samples are available onlinefootnotehttps://zzw922cn.github.io/wesinger/.