# A Neural Parametric Singing Synthesizer

arXiv: https://arxiv.org/abs/1704.03809 · HTML: https://mlsanity.com/paper/1704.03809/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Apr 2017.

## Built on by (most cited first, 14 of 14)

- [DiffSinger: Singing Voice Synthesis via Shallow Diffusion Mechanism](https://mlsanity.com/paper/2105.02446/) (2021)
- [Audio representations for deep learning in sound synthesis: A review](https://mlsanity.com/paper/2201.02490/) (2022)
- [A Melody-Unsupervision Model for Singing Voice Synthesis](https://mlsanity.com/paper/2110.06546/) (2021)
- [WeSinger: Data-augmented Singing Voice Synthesis with Auxiliary Losses](https://mlsanity.com/paper/2203.10750/) (2022)
- [LoopNet: Musical Loop Synthesis Conditioned On Intuitive Musical Parameters](https://mlsanity.com/paper/2105.10371/) (2021)
- [SingAug: Data Augmentation for Singing Voice Synthesis with Cycle-consistent Training Strategy](https://mlsanity.com/paper/2203.17001/) (2022)
- [Differentiable WORLD Synthesizer-based Neural Vocoder With Application To End-To-End Audio Style Transfer](https://mlsanity.com/paper/2208.07282/) (2022)
- [Controllable speech synthesis by learning discrete phoneme-level prosodic representations](https://mlsanity.com/paper/2211.16307/) (2022)
- [YingMusic-SVC: Real-World Robust Zero-Shot Singing Voice Conversion with Flow-GRPO and Singing-Specific Inductive Biases](https://mlsanity.com/paper/2512.04793/) (2025)
- [Rapping-Singing Voice Synthesis based on Phoneme-level Prosody Control](https://mlsanity.com/paper/2111.09146/) (2021)
- [Anyone GAN Sing](https://mlsanity.com/paper/2102.11058/) (2021)
- [Synchronising speech segments with musical beats in Mandarin and English singing](https://mlsanity.com/paper/2106.10045/) (2021)
- [Pitch Preservation In Singing Voice Synthesis](https://mlsanity.com/paper/2110.05033/) (2021)
- [Singing Voice Synthesis Based on a Musical Note Position-Aware Attention Mechanism](https://mlsanity.com/paper/2212.13703/) (2022)

## Abstract (arXiv)

We present a new model for singing synthesis based on a modified version of the WaveNet architecture. Instead of modeling raw waveform, we model features produced by a parametric vocoder that separates the influence of pitch and timbre. This allows conveniently modifying pitch to match any target melody, facilitates training on more modest dataset sizes, and significantly reduces training and generation times. Our model makes frame-wise predictions using mixture density outputs rather than categorical outputs in order to reduce the required parameter count. As we found overfitting to be an issue with the relatively small datasets used in our experiments, we propose a method to regularize the model and make the autoregressive generation process more robust to prediction errors. Using a simple multi-stream architecture, harmonic, aperiodic and voiced/unvoiced components can all be predicted in a coherent manner. We compare our method to existing parametric statistical and state-of-the-art concatenative methods using quantitative metrics and a listening test. While naive implementations of the autoregressive generation algorithm tend to be inefficient, using a smart algorithm we can greatly speed up the process and obtain a system that's competitive in both speed and quality.