# Singing Voice Synthesis with Vibrato Modeling and Latent Energy Representation

arXiv: https://arxiv.org/abs/2211.00996 · HTML: https://mlsanity.com/paper/2211.00996/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Nov 2022.

## Built on by (most cited first, 4 of 4)

- [VibE-SVC: Vibrato Extraction with High-frequency F0 Contour for Singing Voice Conversion](https://mlsanity.com/paper/2505.20794/) (2025)
- [Controllable Singing Voice Synthesis using Phoneme-Level Energy Sequence](https://mlsanity.com/paper/2509.07038/) (2025)
- [Controllable Singing Style Conversion with Boundary-Aware Information Bottleneck](https://mlsanity.com/paper/2604.05526/) (2026)
- [Vibrato Expression Control for Singing Voice Conversion with Improving Independent Control](https://mlsanity.com/paper/2606.17126/) (2026)

## Abstract (arXiv)

This paper proposes an expressive singing voice synthesis system by introducing explicit vibrato modeling and latent energy representation. Vibrato is essential to the naturalness of synthesized sound, due to the inherent characteristics of human singing. Hence, a deep learning-based vibrato model is introduced in this paper to control the vibrato's likeliness, rate, depth and phase in singing, where the vibrato likeliness represents the existence probability of vibrato and it would help improve the singing voice's naturalness. Actually, there is no annotated label about vibrato likeliness in existing singing corpus. We adopt a novel vibrato likeliness labeling method to label the vibrato likeliness automatically. Meanwhile, the power spectrogram of audio contains rich information that can improve the expressiveness of singing. An autoencoder-based latent energy bottleneck feature is proposed for expressive singing voice synthesis. Experimental results on the open dataset NUS48E show that both the vibrato modeling and the latent energy representation could significantly improve the expressiveness of singing voice. The audio samples are shown in the demo website.