# Sequence-to-sequence Singing Synthesis Using the Feed-forward Transformer

arXiv: https://arxiv.org/abs/1910.09989 · HTML: https://mlsanity.com/paper/1910.09989/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Oct 2019.

## Built on by (most cited first, 17 of 17)

- [DiffSinger: Singing Voice Synthesis via Shallow Diffusion Mechanism](https://mlsanity.com/paper/2105.02446/) (2021)
- [CoMoSpeech: One-Step Speech and Singing Voice Synthesis via Consistency Model](https://mlsanity.com/paper/2305.06908/) (2023)
- [Sinsy: A Deep Neural Network-Based Singing Voice Synthesis System](https://mlsanity.com/paper/2108.02776/) (2021)
- [Learning the Beauty in Songs: Neural Singing Voice Beautifier](https://mlsanity.com/paper/2202.13277/) (2022)
- [A Melody-Unsupervision Model for Singing Voice Synthesis](https://mlsanity.com/paper/2110.06546/) (2021)
- [Mandarin Singing Voice Synthesis with Denoising Diffusion Probabilistic Wasserstein GAN](https://mlsanity.com/paper/2209.10446/) (2022)
- [Singing-Tacotron: Global duration control attention and dynamic filter for End-to-end singing voice synthesis](https://mlsanity.com/paper/2202.07907/) (2022)
- [Towards Improving the Expressiveness of Singing Voice Synthesis with BERT Derived Semantic Information](https://mlsanity.com/paper/2308.16836/) (2023)
- [KaraSinger: Score-Free Singing Voice Synthesis with VQ-VAE using Mel-spectrograms](https://mlsanity.com/paper/2110.04005/) (2021)
- [SingAug: Data Augmentation for Singing Voice Synthesis with Cycle-consistent Training Strategy](https://mlsanity.com/paper/2203.17001/) (2022)
- [NNSVS: A Neural Network-Based Singing Voice Synthesis Toolkit](https://mlsanity.com/paper/2210.15987/) (2022)
- [MakeSinger: A Semi-Supervised Training Method for Data-Efficient Singing Voice Synthesis via Classifier-free Diffusion Guidance](https://mlsanity.com/paper/2406.05965/) (2024)
- [SiFiSinger: A High-Fidelity End-to-End Singing Voice Synthesizer based on Source-filter Model](https://mlsanity.com/paper/2410.12536/) (2024)
- [Pitch Preservation In Singing Voice Synthesis](https://mlsanity.com/paper/2110.05033/) (2021)
- [Singing Voice Synthesis Based on a Musical Note Position-Aware Attention Mechanism](https://mlsanity.com/paper/2212.13703/) (2022)
- [Singing voice synthesis based on frame-level sequence-to-sequence models considering vocal timing deviation](https://mlsanity.com/paper/2301.02262/) (2023)
- [ConSinger: Efficient High-Fidelity Singing Voice Generation with Minimal Steps](https://mlsanity.com/paper/2410.15342/) (2024)

## Abstract (arXiv)

We propose a sequence-to-sequence singing synthesizer, which avoids the need for training data with pre-aligned phonetic and acoustic features. Rather than the more common approach of a content-based attention mechanism combined with an autoregressive decoder, we use a different mechanism suitable for feed-forward synthesis. Given that phonetic timings in singing are highly constrained by the musical score, we derive an approximate initial alignment with the help of a simple duration model. Then, using a decoder based on a feed-forward variant of the Transformer model, a series of self-attention and convolutional layers refines the result of the initial alignment to reach the target acoustic features. Advantages of this approach include faster inference and avoiding the exposure bias issues that affect autoregressive models trained by teacher forcing. We evaluate the effectiveness of this model compared to an autoregressive baseline, the importance of self-attention, and the importance of the accuracy of the duration model.