# DeepFry: Identifying Vocal Fry Using Deep Neural Networks

arXiv: https://arxiv.org/abs/2203.17019 · HTML: https://mlsanity.com/paper/2203.17019/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 29 references that are in the map.
- First posted on arXiv 31 Mar 2022.

## Builds on (most cited first, 6 of 6)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://mlsanity.com/paper/2106.07447/) (2021)
- [Dr.VOT : Measuring Positive and Negative Voice Onset Time in the Wild](https://mlsanity.com/paper/1910.13255/) (2019)
- [Corpus Phonetics Tutorial](https://mlsanity.com/paper/1811.05553/) (2018)
- [Data-driven Detection and Analysis of the Patterns of Creaky Voice](https://mlsanity.com/paper/2006.00518/) (2020)

## Built on by (most cited first, 2 of 2)

- [Can a Machine Distinguish High and Low Amount of Social Creak in Speech?](https://mlsanity.com/paper/2410.17028/) (2024)
- [Vibrato Expression Control for Singing Voice Conversion with Improving Independent Control](https://mlsanity.com/paper/2606.17126/) (2026)

## Abstract (arXiv)

Vocal fry or creaky voice refers to a voice quality characterized by irregular glottal opening and low pitch. It occurs in diverse languages and is prevalent in American English, where it is used not only to mark phrase finality, but also sociolinguistic factors and affect. Due to its irregular periodicity, creaky voice challenges automatic speech processing and recognition systems, particularly for languages where creak is frequently used. This paper proposes a deep learning model to detect creaky voice in fluent speech. The model is composed of an encoder and a classifier trained together. The encoder takes the raw waveform and learns a representation using a convolutional neural network. The classifier is implemented as a multi-headed fully-connected network trained to detect creaky voice, voicing, and pitch, where the last two are used to refine creak prediction. The model is trained and tested on speech of American English speakers, annotated for creak by trained phoneticians. We evaluated the performance of our system using two encoders: one is tailored for the task, and the other is based on a state-of-the-art unsupervised representation. Results suggest our best-performing system has improved recall and F1 scores compared to previous methods on unseen data.