# Emotion-Aware Speech Generation with Character-Specific Voices for Comics

arXiv: https://arxiv.org/abs/2509.15253 · HTML: https://mlsanity.com/paper/2509.15253/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 19 of its 26 references that are in the map.
- First posted on arXiv 18 Sep 2025.

## Builds on (most cited first, 19 of 19)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations](https://mlsanity.com/paper/1602.07332/) (2016)
- [An End-to-End Trainable Neural Network for Image-based Sequence Recognition and Its Application to Scene Text Recognition](https://mlsanity.com/paper/1507.05717/) (2015)
- [Scene Graph Generation by Iterative Message Passing](https://mlsanity.com/paper/1701.02426/) (2017)
- [Neural Motifs: Scene Graph Parsing with Global Context](https://mlsanity.com/paper/1711.06640/) (2017)
- [Sketch-based Manga Retrieval using Manga109 Dataset](https://mlsanity.com/paper/1510.04389/) (2015)
- [Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis](https://mlsanity.com/paper/1806.04558/) (2018)
- [Building a Manga Dataset "Manga109" with Annotations for Multimedia Applications](https://mlsanity.com/paper/2005.04425/) (2020)
- [WavJourney: Compositional Audio Creation with Large Language Models](https://mlsanity.com/paper/2307.14335/) (2023)
- [EmoSphere-TTS: Emotional Style and Intensity Modeling via Spherical Emotion Vector for Controllable Emotional Text-to-Speech](https://mlsanity.com/paper/2406.07803/) (2024)
- [Manga109Dialog: A Large-scale Dialogue Dataset for Comics Speaker Detection](https://mlsanity.com/paper/2306.17469/) (2023)
- [WavCraft: Audio Editing and Generation with Large Language Models](https://mlsanity.com/paper/2403.09527/) (2024)
- [Unsupervised Manga Character Re-identification via Face-body and Spatial-temporal Associated Clustering](https://mlsanity.com/paper/2204.04621/) (2022)
- [Zero-Shot Character Identification and Speaker Prediction in Comics via Iterative Multimodal Fusion](https://mlsanity.com/paper/2404.13993/) (2024)
- [Identity-Aware Semi-Supervised Learning for Comic Character Re-Identification](https://mlsanity.com/paper/2308.09096/) (2023)
- [Bridging Paintings and Music -- Exploring Emotion based Music Generation through Paintings](https://mlsanity.com/paper/2409.07827/) (2024)
- [Bridging Discrete and Continuous: A Multimodal Strategy for Complex Emotion Detection](https://mlsanity.com/paper/2409.07901/) (2024)

## Abstract (arXiv)

This paper presents an end-to-end pipeline for generating character-specific, emotion-aware speech from comics. The proposed system takes full comic volumes as input and produces speech aligned with each character's dialogue and emotional state. An image processing module performs character detection, text recognition, and emotion intensity recognition. A large language model performs dialogue attribution and emotion analysis by integrating visual information with the evolving plot context. Speech is synthesized through a text-to-speech model with distinct voice profiles tailored to each character and emotion. This work enables automated voiceover generation for comics, offering a step toward interactive and immersive comic reading experience.