mlsanityData to 14 Oct 2026

Can Prosodic Style Be Inferred from Text Alone? Evidence from Unsupervised Acoustic Clusters

arXiv 2610.05575 ↗First posted 4 Oct 2026cs.CL cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Much of expressive text-to-speech research rests on an untested assumption that written text carries enough information to select an appropriate prosodic style for its delivery. Text-predicted style models improve listener preference, and expressive-appropriateness evaluation presupposes that context constrains style, yet neither measures the assumption itself. This paper tests it as a falsifiable hypothesis against style labels derived from acoustics alone. For each of six speakers in a 1,200-hour conversational corpus, utterances are clustered in the spaces of five speech models, including a prosody-only control, and the cluster of held-out utterances is predicted from twelve text embedding models. Three controls are applied: utterance length is erased from the speech embeddings; accuracy is scored against the majority-class floor of unbalanced clusters rather than uniform chance; and a bag-of-words baseline measures word identity alone. Text predicts the cluster above that floor for all six speakers (+0.111 top-3 accuracy), but bag-of-words achieves three quarters of this. Sentence embeddings add only +0.026, largest for encoders not trained for sentence semantics and reversed by tree-based probes for all others. Acoustic clusters are not compact in text embedding space in any of 360 configurations. The prosody-only space weakens the association for five speakers, but not for the speaker showing it most strongly. Text thus informs these delivery clusters mainly through word choice, whether as a cue to prosody or as a marker of topic and recording situation, and reference-free style selection cannot assume more.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 40 of its 50 references that are in the map. First posted on arXiv 4 Oct 2026.

Builds on

40 of 50 references.

References · 40

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  3. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  4. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  5. 2020Scaling Laws for Neural Language Modelscited by 6,586
  6. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  7. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,307
  8. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  9. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
  10. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,342
  11. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,657
  12. 2020MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformerscited by 1,463
28 more
  1. 2020MPNet: Masked and Permuted Pre-training for Language Understandingcited by 981
  2. 2017Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictionscited by 804
  3. 2020FastSpeech 2: Fast and High-Quality End-to-End Text to Speechcited by 717
  4. 2021SUPERB: Speech processing Universal PERformance Benchmarkcited by 666
  5. 2021Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speechcited by 561
  6. 2020Unsupervised Cross-lingual Representation Learning for Speech Recognitioncited by 501
  7. 2017Tacotron: Towards End-to-End Speech Synthesiscited by 490
  8. 2024Nomic Embed: Training a Reproducible Long Context Text Embeddercited by 265
  9. 2018Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesiscited by 259
  10. 2020Real Time Speech Enhancement in the Waveform Domaincited by 186
  11. 2023StyleTTS 2: Towards Human-Level Text-to-Speech through Style Diffusion and Adversarial Training with Large Speech Language Modelscited by 152
  12. 2020FastPitch: Parallel Text-to-speech with Pitch Predictioncited by 151
  13. 2024BASE TTS: Lessons from building a billion-parameter Text-to-Speech model on 100K hours of datacited by 75
  14. 2022ContentVec: An Improved Self-Supervised Speech Representation by Disentangling Speakerscited by 74
  15. 2023What Do Self-Supervised Speech Models Know About Words?cited by 64
  16. 2018Predicting Expressive Speaking Style From Text In End-To-End Speech Synthesiscited by 32
  17. 2023ProsAudit, a prosodic benchmark for self-supervised speech modelscited by 25
  18. 2023CLAPSpeech: Learning Prosody from Text Context with Contrastive Language-Audio Pre-trainingcited by 18
  19. 2023Speech Self-Supervised Representations Benchmarking: a Case for Larger Probing Headscited by 17
  20. 2023DiffProsody: Diffusion-based Latent Prosody Generation for Expressive Speech Synthesis with Prosody Conditional Adversarial Trainingcited by 16
  21. 2024What Do Self-Supervised Speech and Speaker Models Learn? New Findings From a Cross Model Layer-Wise Analysiscited by 11
  22. 2024Hierarchical Emotion Prediction and Control in Text-to-Speech Synthesiscited by 7
  23. 2023Diverse and Expressive Speech Prosody Prediction with Denoising Diffusion Probabilistic Modelcited by 4
  24. 2024Text-aware and Context-aware Expressive Audiobook Speech Synthesiscited by 3
  25. 2026Evaluating the Expressive Appropriateness of Speech in Rich Contextscited by 3
  26. 2025HiStyle: Hierarchical Style Embedding Predictor for Text-Prompt-Guided Controllable Speech Synthesiscited by 2
  27. 2023Context-aware Coherent Speaking Style Prediction with Hierarchical Transformers for Audiobook Speech Synthesiscited by 1
  28. 2025ProMode: A Speech Prosody Model Conditioned on Acoustic and Textual Inputscited by 1

Built on by

Not cited yet.