# Prominence-aware automatic speech recognition for conversational speech

arXiv: https://arxiv.org/abs/2509.10116 · HTML: https://mlsanity.com/paper/2509.10116/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 4 of its 19 references that are in the map.
- First posted on arXiv 12 Sep 2025.

## Builds on (most cited first, 4 of 4)

- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [Unsupervised Cross-lingual Representation Learning for Speech Recognition](https://mlsanity.com/paper/2006.13979/) (2020)
- [Detection of Prosodic Boundaries in Speech Using Wav2Vec 2.0](https://mlsanity.com/paper/2209.15032/) (2022)
- [Using Kaldi for Automatic Speech Recognition of Conversational Austrian German](https://mlsanity.com/paper/2301.06475/) (2023)

## Abstract (arXiv)

This paper investigates prominence-aware automatic speech recognition (ASR) by combining prominence detection and speech recognition for conversational Austrian German. First, prominence detectors were developed by fine-tuning wav2vec2 models to classify word-level prominence. The detector was then used to automatically annotate prosodic prominence in a large corpus. Based on those annotations, we trained novel prominence-aware ASR systems that simultaneously transcribe words and their prominence levels. The integration of prominence information did not change performance compared to our baseline ASR system, while reaching a prominence detection accuracy of 85.53% for utterances where the recognized word sequence was correct. This paper shows that transformer-based models can effectively encode prosodic information and represents a novel contribution to prosody-enhanced ASR, with potential applications for linguistic research and prosody-informed dialogue systems.