StoryTTS: A Highly Expressive Text-to-Speech Dataset with Rich Textual Expressiveness Annotations
Not on the map. Why?
Abstract (arXiv)
While acoustic expressiveness has long been studied in expressive text-to-speech (ETTS), the inherent expressiveness in text lacks sufficient attention, especially for ETTS of artistic works. In this paper, we introduce StoryTTS, a highly ETTS dataset that contains rich expressiveness both in acoustic and textual perspective, from the recording of a Mandarin storytelling show. A systematic and comprehensive labeling framework is proposed for textual expressiveness. We analyze and define speech-related textual expressiveness in StoryTTS to include five distinct dimensions through linguistics, rhetoric, etc. Then we employ large language models and prompt them with a few manual annotation examples for batch annotation. The resulting corpus contains 61 hours of consecutive and highly prosodic speech equipped with accurate text transcriptions and rich textual expressiveness annotations. Therefore, StoryTTS can aid future ETTS research to fully mine the abundant intrinsic textual and acoustic features. Experiments are conducted to validate that TTS models can generate speech with improved expressiveness when integrating with the annotated textual labels in StoryTTS.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 21 of its 32 references that are in the map. First posted on arXiv 23 Apr 2024.
Builds on
21 of 32 references.
References · 21
- 2017Attention Is All You Needcited by 60,361
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2023GPT-4 Technical Reportcited by 18,396
- 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
- 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
- 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,342
- 2017Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictionscited by 804
- 2020FastSpeech 2: Fast and High-Quality End-to-End Text to Speechcited by 717
- 2019LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speechcited by 649
- 2021Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speechcited by 561
- 2020Unsupervised Cross-lingual Representation Learning for Speech Recognitioncited by 501
9 more
- 2017Tacotron: Towards End-to-End Speech Synthesiscited by 490
- 2019vq-wav2vec: Self-Supervised Learning of Discrete Speech Representationscited by 340
- 2022Is GPT-3 a Good Data Annotator?cited by 208
- 2021Hi-Fi Multi-Speaker English TTS Datasetcited by 60
- 2022VQTTS: High-Fidelity Text-to-Speech Synthesis with Self-Supervised VQ Acoustic Featurecited by 32
- 2022SyntaSpeech: Syntax-Aware Generative Adversarial Text-to-Speechcited by 18
- 2022Unsupervised word-level prosody tagging for controllable speech synthesiscited by 7
- 2021Rich Prosody Diversity Modelling with Phone-level Mixture Density Networkcited by 5
- 2023DSE-TTS: Dual Speaker Embedding for Cross-Lingual Text-to-Speechcited by 5
Built on by
Citing papers per year: 2025: 1; 2026: 1.
2 papers.