# Beyond Words: Towards Effective Modeling of Non-Verbal Vocalizations in ASR

arXiv: https://arxiv.org/abs/2607.01563 · HTML: https://mlsanity.com/paper/2607.01563/

## Key facts

- No citing papers in the map yet (data through 19 Oct 2026).
- Placed in Emotional speech synthesis and recognition, one of 11 directions in Speech and audio language models.
- First posted on arXiv 2 Jul 2026.

## Placed in

- Speech and audio language models / [Emotional speech synthesis and recognition](https://mlsanity.com/speech/emotional-speech-synthesis-and-recognition/)

## Abstract (arXiv)

Modern automatic speech recognition (ASR) systems excel at transcribing lexical content but often omit nonverbal vocalizations (NVs), such as laughter, breaths, coughs, and cries, that carry conversational and affective information. Modeling NVs in ASR is challenging because NV annotations are sparse and highly long-tailed, with frequent categories such as breaths and laughter dominating rarer events such as cries and coughs. We study three data-centric strategies for improving low-resource NV recognition: (1) a two-stage curriculum that first maps all NV events to a generic token and then fine-tunes on target categories; (2) inter-token transfer from high-resource events, such as laughter and breath, to rare events, such as crying; and (3) voice-conversion augmentation with class balancing. Experiments show that shared acoustic structure across vocal events can be exploited to improve rare-category detection while preserving lexical ASR quality.