mlsanityData to 7 Oct 2026

Singing voice synthesis based on frame-level sequence-to-sequence models considering vocal timing deviation

arXiv 2301.02262 ↗First posted 5 Jan 2023eess.AS cs.LG cs.SDcited by 0

Not on the map. Why?

Abstract (arXiv)

This paper proposes singing voice synthesis (SVS) based on frame-level sequence-to-sequence models considering vocal timing deviation. In SVS, it is essential to synchronize the timing of singing with temporal structures represented by scores, taking into account that there are differences between actual vocal timing and note start timing. In many SVS systems including our previous work, phoneme-level score features are converted into frame-level ones on the basis of phoneme boundaries obtained by external aligners to take into account vocal timing deviations. Therefore, the sound quality is affected by the aligner accuracy in this system. To alleviate this problem, we introduce an attention mechanism with frame-level features. In the proposed system, the attention mechanism absorbs alignment errors in phoneme boundaries. Additionally, we evaluate the system with pseudo-phoneme-boundaries defined by heuristic rules based on musical scores when there is no aligner. The experimental results show the effectiveness of the proposed system.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 15 of its 27 references that are in the map. First posted on arXiv 5 Jan 2023.

Builds on

15 of 27 references.

References · 15

  1. 2017Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictionscited by 800
  2. 2020FastSpeech 2: Fast and High-Quality End-to-End Text to Speechcited by 710
  3. 2021Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speechcited by 555
  4. 2017Efficiently Trainable Text-to-Speech System Based on Deep Convolutional Networks with Guided Attentioncited by 57
  5. 2020HiFiSinger: Towards High-Fidelity Neural Singing Voice Synthesiscited by 39
  6. 2020XiaoiceSing: A High-Quality and Integrated Singing Voice Synthesis Systemcited by 33
  7. 2022JETS: Jointly Training FastSpeech2 and HiFi-GAN for End to End Text to Speechcited by 26
  8. 2019Adversarially Trained End-to-end Korean Singing Voice Synthesis Systemcited by 22
  9. 2020ByteSing: A Chinese Singing Voice Synthesis System Using Duration Allocated Encoder-Decoder Acoustic Models and WaveRNN Vocoderscited by 21
  10. 2019Sequence-to-sequence Singing Synthesis Using the Feed-forward Transformercited by 17
  11. 2021Sinsy: A Deep Neural Network-Based Singing Voice Synthesis Systemcited by 14
  12. 2020Adversarially Trained Multi-Singer Sequence-To-Sequence Singing Synthesizercited by 12
3 more
  1. 2021PeriodNet: A non-autoregressive waveform generation model with a structure separating periodic and aperiodic componentscited by 7
  2. 2020Sequence-to-sequence Singing Voice Synthesis with Perceptual Entropy Losscited by 6
  3. 2022Mandarin Singing Voice Synthesis with Denoising Diffusion Probabilistic Wasserstein GANcited by 5

Built on by

Not cited yet.