mlsanityData to 7 Oct 2026

Does a prosody-trained representation help beyond trainable fusion? A parameter-matched study with frozen HuBERT

arXiv 2609.36754 ↗First posted 29 Sep 2026eess.AS cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Explicit prosodic cues may help automatic speech recognition (ASR) of spontaneous speech, but auxiliary representations typically require additional trainable components, making it unclear whether gains come from the auxiliary information or the fusion mechanism. We address this using a frozen HuBERT backbone and a 64-dimensional representation trained to predict log F0, voicing, Delta log F0, log energy, and spectral tilt. We compare a frozen-backbone recognizer (Baseline), trainable fusion with zero auxiliary input (Null), and the same fusion supplied with the learned representation (Learned). Across Buckeye, Switchboard, and AMI IHM, Null reduces WER by 0.71-1.45 points over Baseline, whereas Learned differs from Null by +0.07, -0.09, and +0.00 points, with no significant differences. However, removing or mismatching the representation at inference increases Learned WER. Thus, Learned depends on the representation yet shows no measurable incremental WER benefit over the parameter-matched control.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 13 of its 28 references that are in the map. First posted on arXiv 29 Sep 2026.

Builds on

13 of 28 references.

References · 13

  1. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
  2. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  3. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,827
  4. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,301
  5. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,286
  6. 2019Designing and Interpreting Probes with Control Taskscited by 608
  7. 2022Comparative layer-wise analysis of self-supervised speech modelscited by 155
  8. 2018CREPE: A Convolutional Representation for Pitch Estimationcited by 109
  9. 2022On the Utility of Self-supervised Models for Prosody-related Taskscited by 42
  10. 2023ProsAudit, a prosodic benchmark for self-supervised speech modelscited by 24
  11. 2023Speech Self-Supervised Representations Benchmarking: a Case for Larger Probing Headscited by 16
  12. 2025CASPER: A Large Scale Spontaneous Speech Datasetcited by 3
1 more
  1. 2025Pitch Accent Detection improves Pretrained Automatic Speech Recognitioncited by 1

Built on by

Not cited yet.