# Pitch Preservation In Singing Voice Synthesis

arXiv: https://arxiv.org/abs/2110.05033 · HTML: https://mlsanity.com/paper/2110.05033/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 27 references that are in the map.
- First posted on arXiv 11 Oct 2021.

## Builds on (most cited first, 17 of 17)

- [Unsupervised Domain Adaptation by Backpropagation](https://mlsanity.com/paper/1409.7495/) (2014)
- [WaveNet: A Generative Model for Raw Audio](https://mlsanity.com/paper/1609.03499/) (2016)
- [FastSpeech 2: Fast and High-Quality End-to-End Text to Speech](https://mlsanity.com/paper/2006.04558/) (2020)
- [Multi-band MelGAN: Faster Waveform Generation for High-Quality Text-to-Speech](https://mlsanity.com/paper/2005.05106/) (2020)
- [Mellotron: Multispeaker expressive voice synthesis by conditioning on rhythm, pitch and global style tokens](https://mlsanity.com/paper/1910.11997/) (2019)
- [XiaoiceSing: A High-Quality and Integrated Singing Voice Synthesis System](https://mlsanity.com/paper/2006.06261/) (2020)
- [Adversarially Trained End-to-end Korean Singing Voice Synthesis System](https://mlsanity.com/paper/1908.01919/) (2019)
- [ByteSing: A Chinese Singing Voice Synthesis System Using Duration Allocated Encoder-Decoder Acoustic Models and WaveRNN Vocoders](https://mlsanity.com/paper/2004.11012/) (2020)
- [Sequence-to-sequence Singing Synthesis Using the Feed-forward Transformer](https://mlsanity.com/paper/1910.09989/) (2019)
- [PitchNet: Unsupervised Singing Voice Conversion with Pitch Adversarial Network](https://mlsanity.com/paper/1912.01852/) (2019)
- [A Neural Parametric Singing Synthesizer](https://mlsanity.com/paper/1704.03809/) (2017)
- [DurIAN-SC: Duration Informed Attention Network based Singing Voice Conversion System](https://mlsanity.com/paper/2008.03009/) (2020)
- [Adversarially Trained Multi-Singer Sequence-To-Sequence Singing Synthesizer](https://mlsanity.com/paper/2006.10317/) (2020)
- [Singing Voice Synthesis Using Deep Autoregressive Neural Networks for Acoustic Modeling](https://mlsanity.com/paper/1906.08977/) (2019)
- [Fast and High-Quality Singing Voice Synthesis System based on Convolutional Neural Networks](https://mlsanity.com/paper/1910.11690/) (2019)
- [Data Efficient Voice Cloning from Noisy Samples with Domain Adversarial Training](https://mlsanity.com/paper/2008.04265/) (2020)
- [Learn2Sing: Target Speaker Singing Voice Synthesis by learning from a Singing Teacher](https://mlsanity.com/paper/2011.08467/) (2020)

## Abstract (arXiv)

Suffering from limited singing voice corpus, existing singing voice synthesis (SVS) methods that build encoder-decoder neural networks to directly generate spectrogram could lead to out-of-tune issues during the inference phase. To attenuate these issues, this paper presents a novel acoustic model with independent pitch encoder and phoneme encoder, which disentangles the phoneme and pitch information from music score to fully utilize the corpus. Specifically, according to equal temperament theory, the pitch encoder is constrained by a pitch metric loss that maps distances between adjacent input pitches into corresponding frequency multiples between the encoder outputs. For the phoneme encoder, based on the analysis that same phonemes corresponding to varying pitches can produce similar pronunciations, this encoder is followed by an adversarially trained pitch classifier to enforce the identical phonemes with different pitches mapping into the same phoneme feature space. By these means, the sparse phonemes and pitches in original input spaces can be transformed into more compact feature spaces respectively, where the same elements cluster closely and cooperate mutually to enhance synthesis quality. Then, the outputs of the two encoders are summed together to pass through the following decoder in the acoustic model. Experimental results indicate that the proposed approaches can characterize intrinsic structure between pitch inputs to obtain better pitch synthesis accuracy and achieve superior singing synthesis performance against the advanced baseline system.