# Synthesizing Personalized Non-speech Vocalization from Discrete Speech Representations

arXiv: https://arxiv.org/abs/2206.12662 · HTML: https://mlsanity.com/paper/2206.12662/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 14 references that are in the map.
- First posted on arXiv 25 Jun 2022.

## Builds on (most cited first, 5 of 5)

- [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://mlsanity.com/paper/2106.07447/) (2021)
- [FastSpeech 2: Fast and High-Quality End-to-End Text to Speech](https://mlsanity.com/paper/2006.04558/) (2020)
- [The ICML 2022 Expressive Vocalizations Workshop and Competition: Recognizing, Generating, and Personalizing Vocal Bursts](https://mlsanity.com/paper/2205.01780/) (2022)
- [HooliGAN: Robust, High Quality Neural Vocoding](https://mlsanity.com/paper/2008.02493/) (2020)
- [Generating Diverse Realistic Laughter for Interactive Art](https://mlsanity.com/paper/2111.03146/) (2021)

## Built on by (most cited first, 2 of 2)

- [EmoGator: A New Open Source Vocal Burst Dataset with Baseline Machine Learning Classification Methodologies](https://mlsanity.com/paper/2301.00508/) (2023)
- [The NeurIPS 2023 Machine Learning for Audio Workshop: Affective Audio Benchmarks and Novel Data](https://mlsanity.com/paper/2403.14048/) (2024)

## Abstract (arXiv)

We formulated non-speech vocalization (NSV) modeling as a text-to-speech task and verified its viability. Specifically, we evaluated the phonetic expressivity of HUBERT speech units on NSVs and verified our model's ability to control over speaker timbre even though the training data is speaker few-shot. In addition, we substantiated that the heterogeneity in recording conditions is the major obstacle for NSV modeling. Finally, we discussed five improvements over our method for future research. Audio samples of synthesized NSVs are available on our demo page: https://resemble-ai.github.io/reLaugh.