mlsanityData to 14 Oct 2026

Emotion-Aware Speech Generation with Character-Specific Voices for Comics

arXiv 2509.15253 ↗First posted 18 Sep 2025cs.SD cs.AI cs.MMcited by 0

Not on the map. Why?

Abstract (arXiv)

This paper presents an end-to-end pipeline for generating character-specific, emotion-aware speech from comics. The proposed system takes full comic volumes as input and produces speech aligned with each character's dialogue and emotional state. An image processing module performs character detection, text recognition, and emotion intensity recognition. A large language model performs dialogue attribution and emotion analysis by integrating visual information with the evolving plot context. Speech is synthesized through a text-to-speech model with distinct voice profiles tailored to each character and emotion. This work enables automated voiceover generation for comics, offering a step toward interactive and immersive comic reading experience.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 19 of its 26 references that are in the map. First posted on arXiv 18 Sep 2025.

Builds on

19 of 26 references.

References · 19

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  3. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  4. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,656
  5. 2015An End-to-End Trainable Neural Network for Image-based Sequence Recognition and Its Application to Scene Text Recognitioncited by 520
  6. 2017Scene Graph Generation by Iterative Message Passingcited by 462
  7. 2017Neural Motifs: Scene Graph Parsing with Global Contextcited by 414
  8. 2015Sketch-based Manga Retrieval using Manga109 Datasetcited by 329
  9. 2018Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesiscited by 282
  10. 2020Building a Manga Dataset "Manga109" with Annotations for Multimedia Applicationscited by 38
  11. 2023WavJourney: Compositional Audio Creation with Large Language Modelscited by 27
  12. 2024EmoSphere-TTS: Emotional Style and Intensity Modeling via Spherical Emotion Vector for Controllable Emotional Text-to-Speechcited by 24
7 more
  1. 2023Manga109Dialog: A Large-scale Dialogue Dataset for Comics Speaker Detectioncited by 14
  2. 2024WavCraft: Audio Editing and Generation with Large Language Modelscited by 14
  3. 2022Unsupervised Manga Character Re-identification via Face-body and Spatial-temporal Associated Clusteringcited by 9
  4. 2024Zero-Shot Character Identification and Speaker Prediction in Comics via Iterative Multimodal Fusioncited by 9
  5. 2023Identity-Aware Semi-Supervised Learning for Comic Character Re-Identificationcited by 8
  6. 2024Bridging Paintings and Music -- Exploring Emotion based Music Generation through Paintingscited by 6
  7. 2024Bridging Discrete and Continuous: A Multimodal Strategy for Complex Emotion Detectioncited by 6

Built on by

Not cited yet.