# The Use of Voice Source Features for Sung Speech Recognition

arXiv: https://arxiv.org/abs/2102.10376 · HTML: https://mlsanity.com/paper/2102.10376/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 5 of its 38 references that are in the map.
- First posted on arXiv 20 Feb 2021.

## Builds on (most cited first, 5 of 5)

- [Joint Robust Voicing Detection and Pitch Estimation Based on Residual Harmonics](https://mlsanity.com/paper/2001.00459/) (2019)
- [The CAPIO 2017 Conversational Speech Recognition System](https://mlsanity.com/paper/1801.00059/) (2017)
- [A Comparative Study of Pitch Extraction Algorithms on a Large Variety of Singing Sounds](https://mlsanity.com/paper/1912.12609/) (2019)
- [Mandarin tone modeling using recurrent neural networks](https://mlsanity.com/paper/1711.01946/) (2017)
- [Transcribing Lyrics From Commercial Song Audio: The First Step Towards Singing Content Processing](https://mlsanity.com/paper/1804.05306/) (2018)

## Abstract (arXiv)

In this paper, we ask whether vocal source features (pitch, shimmer, jitter, etc) can improve the performance of automatic sung speech recognition, arguing that conclusions previously drawn from spoken speech studies may not be valid in the sung speech domain. We first use a parallel singing/speaking corpus (NUS-48E) to illustrate differences in sung vs spoken voicing characteristics including pitch range, syllables duration, vibrato, jitter and shimmer. We then use this analysis to inform speech recognition experiments on the sung speech DSing corpus, using a state of the art acoustic model and augmenting conventional features with various voice source parameters. Experiments are run with three standard (increasingly large) training sets, DSing1 (15.1 hours), DSing3 (44.7 hours) and DSing30 (149.1 hours). Pitch combined with degree of voicing produces a significant decrease in WER from 38.1% to 36.7% when training with DSing1 however smaller decreases in WER observed when training with the larger more varied DSing3 and DSing30 sets were not seen to be statistically significant. Voicing quality characteristics did not improve recognition performance although analysis suggests that they do contribute to an improved discrimination between voiced/unvoiced phoneme pairs.