# WHISTRESS: Enriching Transcriptions with Sentence Stress Detection

arXiv: https://arxiv.org/abs/2505.19103 · HTML: https://mlsanity.com/paper/2505.19103/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 30 references that are in the map.
- First posted on arXiv 25 May 2025.

## Builds on (most cited first, 15 of 15)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [Bidirectional LSTM-CRF Models for Sequence Tagging](https://mlsanity.com/paper/1508.01991/) (2015)
- [TinyStories: How Small Can Language Models Be and Still Speak Coherent English?](https://mlsanity.com/paper/2305.07759/) (2023)
- [WhisperX: Time-Accurate Speech Transcription of Long-Form Audio](https://mlsanity.com/paper/2303.00747/) (2023)
- [Layer-wise Analysis of a Self-supervised Speech Representation Model](https://mlsanity.com/paper/2107.04734/) (2021)
- [EXPRESSO: A Benchmark and Analysis of Discrete Expressive Speech Resynthesis](https://mlsanity.com/paper/2308.05725/) (2023)
- [Fine-Tuning is Fine, if Calibrated](https://mlsanity.com/paper/2409.16223/) (2024)
- [EmphAssess : a Prosodic Benchmark on Assessing Emphasis Transfer in Speech-to-Speech Models](https://mlsanity.com/paper/2312.14069/) (2023)
- [BERT, can HE predict contrastive focus? Predicting and controlling prominence in neural TTS using a language model](https://mlsanity.com/paper/2207.01718/) (2022)
- [Crowdsourced and Automatic Speech Prominence Estimation](https://mlsanity.com/paper/2310.08464/) (2023)
- [Hierarchical Representation of Prosody for Statistical Speech Synthesis](https://mlsanity.com/paper/1510.01949/) (2015)
- [Non-verbal information in spontaneous speech -- towards a new framework of analysis](https://mlsanity.com/paper/2403.03522/) (2024)

## Built on by (most cited first, 7 of 7)

- [StressTest: Can YOUR Speech LM Handle the Stress?](https://mlsanity.com/paper/2505.22765/) (2025)
- [Probing the Hidden Talent of ASR Foundation Models for L2 English Oral Assessment](https://mlsanity.com/paper/2510.16387/) (2025)
- [StressTransfer: Stress-Aware Speech-to-Speech Translation with Emphasis Preservation](https://mlsanity.com/paper/2510.13194/) (2025)
- [Knowing What to Stress: A Discourse-Conditioned Text-to-Speech Benchmark](https://mlsanity.com/paper/2604.10580/) (2026)
- [Do Speech Emphasis Models Generalize across Languages and Emotions?](https://mlsanity.com/paper/2606.27717/) (2026)
- [Unlocking Speech-Text Compositional Powers: Instruction-Following Speech Language Models without Instruction Tuning](https://mlsanity.com/paper/2607.02214/) (2026)
- [EmphTTS: an emphasis-control TTS with reinforcement learning](https://mlsanity.com/paper/2609.27599/) (2026)

## Abstract (arXiv)

Spoken language conveys meaning not only through words but also through intonation, emotion, and emphasis. Sentence stress, the emphasis placed on specific words within a sentence, is crucial for conveying speaker intent and has been extensively studied in linguistics. In this work, we introduce WHISTRESS, an alignment-free approach for enhancing transcription systems with sentence stress detection. To support this task, we propose TINYSTRESS-15K, a scalable, synthetic training data for the task of sentence stress detection which resulted from a fully automated dataset creation process. We train WHISTRESS on TINYSTRESS-15K and evaluate it against several competitive baselines. Our results show that WHISTRESS outperforms existing methods while requiring no additional input priors during training or inference. Notably, despite being trained on synthetic data, WHISTRESS demonstrates strong zero-shot generalization across diverse benchmarks. Project page: https://pages.cs.huji.ac.il/adiyoss-lab/whistress.