Prosody Analysis of Audiobooks
Not on the map. Why?
Abstract (arXiv)
Recent advances in text-to-speech have made it possible to generate natural-sounding audio from text. However, audiobook narrations involve dramatic vocalizations and intonations by the reader, with greater reliance on emotions, dialogues, and descriptions in the narrative. Using our dataset of 93 aligned book-audiobook pairs, we present improved models for prosody prediction properties (pitch, volume, and rate of speech) from narrative text using language modeling. Our predicted prosody attributes correlate much better with human audiobook readings than results from a state-of-the-art commercial TTS system: our predicted pitch shows a higher correlation with human reading for 22 out of the 24 books, while our predicted volume attribute proves more similar to human reading for 23 out of the 24 books. Finally, we present a human evaluation study to quantify the extent that people prefer prosody-enhanced audiobook readings over commercial text-to-speech systems.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 33 of its 55 references that are in the map. First posted on arXiv 10 Oct 2023.
Builds on
33 of 55 references.
References · 33
- 2016WaveNet: A Generative Model for Raw Audiocited by 1,710
- 2020MPNet: Masked and Permuted Pre-training for Language Understandingcited by 973
- 2017Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictionscited by 800
- 2020Stanza: A Python Natural Language Processing Toolkit for Many Human Languagescited by 719
- 2020FastSpeech 2: Fast and High-Quality End-to-End Text to Speechcited by 710
- 2017Tacotron: Towards End-to-End Speech Synthesiscited by 487
- 2018Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesiscited by 257
- 2018Constituency Parsing with a Self-Attentive Encodercited by 196
- 2018Towards End-to-End Prosody Transfer for Expressive Speech Synthesis with Tacotroncited by 171
- 2020FastPitch: Parallel Text-to-speech with Pitch Predictioncited by 150
- 2018Hierarchical Generative Modeling for Controllable Speech Synthesiscited by 93
- 2017Deep Voice: Real-time Neural Text-to-Speechcited by 83
21 more
- 2017Deep Voice 2: Multi-Speaker Neural Text-to-Speechcited by 75
- 2017Deep Voice 3: Scaling Text-to-Speech with Convolutional Sequence Learningcited by 67
- 2020Non-Attentive Tacotron: Robust and Controllable Neural TTS Synthesis Including Unsupervised Duration Modelingcited by 65
- 2020Parallel Tacotron: Non-Autoregressive and Controllable TTScited by 49
- 2021"Let Your Characters Tell Their Story": A Dataset for Character-Centric Narrative Understandingcited by 48
- 2020Wave-Tacotron: Spectrogram-free end-to-end text-to-speech synthesiscited by 38
- 2019DurIAN: Duration Informed Attention Network For Multimodal Synthesiscited by 33
- 2021Parallel Tacotron 2: A Non-Autoregressive Neural TTS Model with Differentiable Duration Modelingcited by 31
- 2021LightSpeech: Lightweight and Fast Text to Speech with Neural Architecture Searchcited by 27
- 2020Screenplay Summarization Using Latent Narrative Structurecited by 22
- 2013From Once Upon a Time to Happily Ever After: Tracking Emotions in Novels and Fairy Talescited by 16
- 2020SpeedySpeech: Efficient Neural Speech Synthesiscited by 15
- 2020Modelling Suspense in Short Stories as Uncertainty Reduction over Neural Representationcited by 13
- 2021Emotional Prosody Control for Speech Generationcited by 12
- 2019Multi-reference Tacotron by Intercross Training for Style Disentangling,Transfer and Control in Speech Synthesiscited by 11
- 2021Memory and Knowledge Augmented Language Models for Inferring Salience in Long-Form Storiescited by 11
- 2020What time is it? Temporal Analysis of Novelscited by 6
- 2020Modeling Event Salience in Narratives via Barthes' Cardinal Functionscited by 5
- 2020Chapter Captor: Text Segmentation in Novelscited by 4
- 2015Information retrieval in folktales using natural language processingcited by 2
- 2020Pitchtron: Towards audiobook generation from ordinary people's voicescited by 1
Built on by
Citing papers per year: 2025: 2; 2026: 1.
3 papers.