mlsanityData to 7 Oct 2026

ProsAudit, a prosodic benchmark for self-supervised speech models

arXiv 2302.12057 ↗First posted 23 Feb 2023cs.CL cs.SD eess.AScited by 24

Not on the map. Why?

Abstract (arXiv)

We present ProsAudit, a benchmark in English to assess structural prosodic knowledge in self-supervised learning (SSL) speech models. It consists of two subtasks, their corresponding metrics, and an evaluation dataset. In the protosyntax task, the model must correctly identify strong versus weak prosodic boundaries. In the lexical task, the model needs to correctly distinguish between pauses inserted between words and within words. We also provide human evaluation scores on this benchmark. We evaluated a series of SSL models and found that they were all able to perform above chance on both tasks, even when evaluated on an unseen language. However, non-native models performed significantly worse than native ones on the lexical task, highlighting the importance of lexical knowledge in this task. We also found a clear effect of size with models trained on more data performing better in the two subtasks.

Key facts. Cited by 24 papers in the map (data through 7 Oct 2026). Builds on 12 of its 21 references that are in the map. First posted on arXiv 23 Feb 2023.

Builds on

12 of 21 references.

References · 12

  1. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  2. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  3. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,301
  4. 2021SUPERB: Speech processing Universal PERformance Benchmarkcited by 652
  5. 2019Libri-Light: A Benchmark for ASR with Limited or No Supervisioncited by 462
  6. 2021Generative Spoken Language Modeling from Raw Audiocited by 336
  7. 2021Text-Free Prosody-Aware Generative Spoken Language Modelingcited by 119
  8. 2020Unsupervised pretraining transfers well across languagescited by 100
  9. 2020The Zero Resource Speech Benchmark 2021: Metrics and baselines for unsupervised spoken language modelingcited by 100
  10. 2021The Zero Resource Speech Challenge 2021: Spoken language modellingcited by 47
  11. 2017The Zero Resource Speech Challenge 2017cited by 45
  12. 2022On the Utility of Self-supervised Models for Prosody-related Taskscited by 42

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 2; 2025: 12; 2026: 9.

24 papers.

Sort:

By area:

  1. 2025On The Landscape of Spoken Language Models: A Comprehensive Surveycited by 116
  2. 2025How Linguistics Learned to Stop Worrying and Love the Language Modelscited by 50
  3. 2024Salmon: A Suite for Acoustic Language Model Evaluationcited by 24
  4. 2023EmphAssess : a Prosodic Benchmark on Assessing Emphasis Transfer in Speech-to-Speech Modelscited by 21
  5. 2024Speech is More Than Words: Do Speech-to-Text Translation Systems Leverage Prosody?cited by 13
  6. 2025SpidR: Learning Fast and Stable Linguistic Units for Spoken Language Models Without Supervisioncited by 6
  7. 2025Exploring the Effect of Segmentation and Vocabulary Size on Speech Tokenization for Speech Language Modelscited by 4
  8. 2025Prosodic Structure Beyond Lexical Content: A Study of Self-Supervised Learningcited by 4
  9. 2025Representation of perceived prosodic similarity of conversational feedbackcited by 1
  10. 2025Pitch Accent Detection improves Pretrained Automatic Speech Recognitioncited by 1
  11. 2025Benchmarking Prosody Encoding in Discrete Speech Tokenscited by 1
  12. 2025Language models as tools for investigating the distinction between possible and impossible natural languagescited by 0
12 more
  1. 2025Large language models have learned to use languagecited by 0
  2. 2025Large language models are not about natural languagecited by 0
  3. 2025Linguists should learn to love speech-based deep learning modelscited by 0
  4. 2026Are Language Models Models?cited by 0
  5. 2026Across the Levels of Analysis: Explaining Predictive Processing in Humans Requires More Than Machine-Estimated Probabilitiescited by 0
  6. 2026You Can't Fight in Here! This is BBS!cited by 0
  7. 2026Knowing What to Stress: A Discourse-Conditioned Text-to-Speech Benchmarkcited by 0
  8. 2026Not-So-Strange Love: Language Models and Generative Linguistic Theories are More Compatible than They Appearcited by 0
  9. 2026Do Speech Emphasis Models Generalize across Languages and Emotions?cited by 0
  10. 2026Prosody-driven Jailbreaks in Audio LLMs: A Controlled Study and Mechanistic Analysiscited by 0
  11. 2026Language Discrimination Improves Linguistic Learning in Multilingual Speech Modelscited by 0
  12. 2026Does a prosody-trained representation help beyond trainable fusion? A parameter-matched study with frozen HuBERTcited by 0