# Towards an Interpretable Representation of Speaker Identity via Perceptual Voice Qualities

arXiv: https://arxiv.org/abs/2310.02497 · HTML: https://mlsanity.com/paper/2310.02497/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 25 references that are in the map.
- First posted on arXiv 4 Oct 2023.

## Builds on (most cited first, 6 of 6)

- [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://mlsanity.com/paper/2106.07447/) (2021)
- [Speaker Recognition Based on Deep Learning: An Overview](https://mlsanity.com/paper/2012.00931/) (2020)
- [Emotion Intensity and its Control for Emotional Voice Conversion](https://mlsanity.com/paper/2201.03967/) (2022)
- [Robust Disentangled Variational Speech Representation Learning for Zero-shot Voice Conversion](https://mlsanity.com/paper/2203.16705/) (2022)
- [Speaker-Independent Acoustic-to-Articulatory Speech Inversion](https://mlsanity.com/paper/2302.06774/) (2023)
- [Deep Neural Network for Automatic Assessment of Dysphonia](https://mlsanity.com/paper/2202.12957/) (2022)

## Built on by (most cited first, 3 of 3)

- [Coding Speech through Vocal Tract Kinematics](https://mlsanity.com/paper/2406.12998/) (2024)
- [PerMod: Perceptually Grounded Voice Modification with Latent Diffusion Models](https://mlsanity.com/paper/2312.08494/) (2023)
- [Speech After Gender: A Trans-Feminine Perspective on Next Steps for Speech Science and Technology](https://mlsanity.com/paper/2407.07235/) (2024)

## Abstract (arXiv)

Unlike other data modalities such as text and vision, speech does not lend itself to easy interpretation. While lay people can understand how to describe an image or sentence via perception, non-expert descriptions of speech often end at high-level demographic information, such as gender or age. In this paper, we propose a possible interpretable representation of speaker identity based on perceptual voice qualities (PQs). By adding gendered PQs to the pathology-focused Consensus Auditory-Perceptual Evaluation of Voice (CAPE-V) protocol, our PQ-based approach provides a perceptual latent space of the character of adult voices that is an intermediary of abstraction between high-level demographics and low-level acoustic, physical, or learned representations. Contrary to prior belief, we demonstrate that these PQs are hearable by ensembles of non-experts, and further demonstrate that the information encoded in a PQ-based representation is predictable by various speech representations.