# Improving Speech Prosody of Audiobook Text-to-Speech Synthesis with Acoustic and Textual Contexts

arXiv: https://arxiv.org/abs/2211.02336 · HTML: https://mlsanity.com/paper/2211.02336/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Nov 2022.

## Built on by (most cited first, 6 of 6)

- [BASE TTS: Lessons from building a billion-parameter Text-to-Speech model on 100K hours of data](https://mlsanity.com/paper/2402.08093/) (2024)
- [MCDubber: Multimodal Context-Aware Expressive Video Dubbing](https://mlsanity.com/paper/2408.11593/) (2024)
- [Improving Audio Codec-based Zero-Shot Text-to-Speech Synthesis with Multi-Modal Context and Large Language Model](https://mlsanity.com/paper/2406.03706/) (2024)
- [AutoStyle-TTS: Retrieval-Augmented Generation based Automatic Style Matching Text-to-Speech Synthesis](https://mlsanity.com/paper/2504.10309/) (2025)
- [Comparative Evaluation of Expressive Japanese Character Text-to-Speech with VITS and Style-BERT-VITS2](https://mlsanity.com/paper/2505.17320/) (2025)
- [S-VoCAL: A Dataset and Evaluation Framework for Inferring Speaking Voice Character Attributes in Literature](https://mlsanity.com/paper/2603.00958/) (2026)

## Abstract (arXiv)

We present a multi-speaker Japanese audiobook text-to-speech (TTS) system that leverages multimodal context information of preceding acoustic context and bilateral textual context to improve the prosody of synthetic speech. Previous work either uses unilateral or single-modality context, which does not fully represent the context information. The proposed method uses an acoustic context encoder and a textual context encoder to aggregate context information and feeds it to the TTS model, which enables the model to predict context-dependent prosody. We conducted comprehensive objective and subjective evaluations on a multi-speaker Japanese audiobook dataset. Experimental results demonstrate that the proposed method significantly outperforms two previous works. Additionally, we present insights about the different choices of context - modalities, lateral information and length - for audiobook TTS that have never been discussed in the literature before.