Representation of perceived prosodic similarity of conversational feedback
Not on the map. Why?
Abstract (arXiv)
Vocal feedback (e.g., `mhm', `yeah', `okay') is an important component of spoken dialogue and is crucial to ensuring common ground in conversational systems. The exact meaning of such feedback is conveyed through both lexical and prosodic form. In this work, we investigate the perceived prosodic similarity of vocal feedback with the same lexical form, and to what extent existing speech representations reflect such similarities. A triadic comparison task with recruited participants is used to measure perceived similarity of feedback responses taken from two different datasets. We find that spectral and self-supervised speech representations encode prosody better than extracted pitch features, especially in the case of feedback from the same speaker. We also find that it is possible to further condense and align the representations to human perception through contrastive learning.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 13 of its 36 references that are in the map. First posted on arXiv 19 May 2025.
Builds on
13 of 36 references.
References · 13
- 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
- 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
- 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,342
- 2021W2v-BERT: Combining Contrastive Learning and Masked Language Modeling for Self-Supervised Speech Pre-Trainingcited by 347
- 2021Layer-wise Analysis of a Self-supervised Speech Representation Modelcited by 298
- 2021A Fine-tuned Wav2vec 2.0/HuBERT Benchmark For Speech Emotion Recognition, Speaker Verification and Spoken Language Understandingcited by 82
- 2022On the Utility of Self-supervised Models for Prosody-related Taskscited by 42
- 2023ProsAudit, a prosodic benchmark for self-supervised speech modelscited by 25
- 2018Fast Spectrogram Inversion using Multi-head Convolutional Neural Networkscited by 22
- 2023What Can an Accent Identifier Learn? Probing Phonetic and Prosodic Information in a Wav2vec2-based Accent Identification Modelcited by 18
- 2024A layer-wise analysis of Mandarin and English suprasegmentals in SSL speech modelscited by 18
- 2021Speech BERT Embedding For Improving Prosody in Neural TTScited by 11
1 more
- 2023What does BERT learn about prosody?cited by 4
Built on by
Citing papers per year: 2026: 1.
1 paper.