# ByteSing: A Chinese Singing Voice Synthesis System Using Duration Allocated Encoder-Decoder Acoustic Models and WaveRNN Vocoders

arXiv: https://arxiv.org/abs/2004.11012 · HTML: https://mlsanity.com/paper/2004.11012/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Apr 2020.

## Built on by (most cited first, 21 of 21)

- [DiffSinger: Singing Voice Synthesis via Shallow Diffusion Mechanism](https://mlsanity.com/paper/2105.02446/) (2021)
- [Sinsy: A Deep Neural Network-Based Singing Voice Synthesis System](https://mlsanity.com/paper/2108.02776/) (2021)
- [RMSSinger: Realistic-Music-Score based Singing Voice Synthesis](https://mlsanity.com/paper/2305.10686/) (2023)
- [WeSinger: Data-augmented Singing Voice Synthesis with Auxiliary Losses](https://mlsanity.com/paper/2203.10750/) (2022)
- [LDM-SVC: Latent Diffusion Model Based Zero-Shot Any-to-Any Singing Voice Conversion with Singer Guidance](https://mlsanity.com/paper/2406.05325/) (2024)
- [RefineGAN: Universally Generating Waveform Better than Ground Truth with Highly Accurate Pitch and Intensity Responses](https://mlsanity.com/paper/2111.00962/) (2021)
- [Singing-Tacotron: Global duration control attention and dynamic filter for End-to-end singing voice synthesis](https://mlsanity.com/paper/2202.07907/) (2022)
- [Towards Improving the Expressiveness of Singing Voice Synthesis with BERT Derived Semantic Information](https://mlsanity.com/paper/2308.16836/) (2023)
- [KaraSinger: Score-Free Singing Voice Synthesis with VQ-VAE using Mel-spectrograms](https://mlsanity.com/paper/2110.04005/) (2021)
- [SingAug: Data Augmentation for Singing Voice Synthesis with Cycle-consistent Training Strategy](https://mlsanity.com/paper/2203.17001/) (2022)
- [NNSVS: A Neural Network-Based Singing Voice Synthesis Toolkit](https://mlsanity.com/paper/2210.15987/) (2022)
- [SiFiSinger: A High-Fidelity End-to-End Singing Voice Synthesizer based on Source-filter Model](https://mlsanity.com/paper/2410.12536/) (2024)
- [Rapping-Singing Voice Synthesis based on Phoneme-level Prosody Control](https://mlsanity.com/paper/2111.09146/) (2021)
- [BiSinger: Bilingual Singing Voice Synthesis](https://mlsanity.com/paper/2309.14089/) (2023)
- [LCM-SVC: Latent Diffusion Model Based Singing Voice Conversion with Inference Acceleration via Latent Consistency Distillation](https://mlsanity.com/paper/2408.12354/) (2024)
- [Karaoker: Alignment-free singing voice synthesis with speech training data](https://mlsanity.com/paper/2204.04127/) (2022)
- [Synchronising speech segments with musical beats in Mandarin and English singing](https://mlsanity.com/paper/2106.10045/) (2021)
- [Pitch Preservation In Singing Voice Synthesis](https://mlsanity.com/paper/2110.05033/) (2021)
- [Singing Voice Synthesis Based on a Musical Note Position-Aware Attention Mechanism](https://mlsanity.com/paper/2212.13703/) (2022)
- [Singing voice synthesis based on frame-level sequence-to-sequence models considering vocal timing deviation](https://mlsanity.com/paper/2301.02262/) (2023)
- [Enhancing the vocal range of single-speaker singing voice synthesis with melody-unsupervised pre-training](https://mlsanity.com/paper/2309.00284/) (2023)

## Abstract (arXiv)

This paper presents ByteSing, a Chinese singing voice synthesis (SVS) system based on duration allocated Tacotron-like acoustic models and WaveRNN neural vocoders. Different from the conventional SVS models, the proposed ByteSing employs Tacotron-like encoder-decoder structures as the acoustic models, in which the CBHG models and recurrent neural networks (RNNs) are explored as encoders and decoders respectively. Meanwhile an auxiliary phoneme duration prediction model is utilized to expand the input sequence, which can enhance the model controllable capacity, model stability and tempo prediction accuracy. WaveRNN neural vocoders are also adopted as neural vocoders to further improve the voice quality of synthesized songs. Both objective and subjective experimental results prove that the SVS method proposed in this paper can produce quite natural, expressive and high-fidelity songs by improving the pitch and spectrogram prediction accuracy and the models using attention mechanism can achieve best performance.