# Deep CNN-based Speech Balloon Detection and Segmentation for Comic Books

arXiv: https://arxiv.org/abs/1902.08137 · HTML: https://mlsanity.com/paper/1902.08137/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Feb 2019.

## Built on by (most cited first, 4 of 4)

- [One missing piece in Vision and Language: A Survey on Comics Understanding](https://mlsanity.com/paper/2409.09502/) (2024)
- [Comics Datasets Framework: Mix of Comics datasets for detection benchmarking](https://mlsanity.com/paper/2407.03540/) (2024)
- [Comics for Everyone: Generating Accessible Text Descriptions for Comic Strips](https://mlsanity.com/paper/2310.00698/) (2023)
- [How Panel Layouts Define Manga: Insights from Visual Ablation Experiments](https://mlsanity.com/paper/2412.19141/) (2024)

## Abstract (arXiv)

We develop a method for the automated detection and segmentation of speech balloons in comic books, including their carrier and tails. Our method is based on a deep convolutional neural network that was trained on annotated pages of the Graphic Narrative Corpus. More precisely, we are using a fully convolutional network approach inspired by the U-Net architecture, combined with a VGG-16 based encoder. The trained model delivers state-of-the-art performance with an F1-score of over 0.94. Qualitative results suggest that wiggly tails, curved corners, and even illusory contours do not pose a major problem. Furthermore, the model has learned to distinguish speech balloons from captions. We compare our model to earlier results and discuss some possible applications.