# Toward accessible comics for blind and low vision readers

arXiv: https://arxiv.org/abs/2407.08248 · HTML: https://mlsanity.com/paper/2407.08248/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 49 references that are in the map.
- First posted on arXiv 11 Jul 2024.

## Builds on (most cited first, 17 of 17)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Large Language Models: A Survey](https://mlsanity.com/paper/2402.06196/) (2024)
- [A Systematic Survey of Prompt Engineering in Large Language Models: Techniques and Applications](https://mlsanity.com/paper/2402.07927/) (2024)
- [Augmented SBERT: Data Augmentation Method for Improving Bi-Encoders for Pairwise Sentence Scoring Tasks](https://mlsanity.com/paper/2010.08240/) (2020)
- [The Amazing Mysteries of the Gutter: Drawing Inferences Between Panels in Comic Book Narratives](https://mlsanity.com/paper/1611.05118/) (2016)
- [Exploring the Impact of Table-to-Text Methods on Augmenting LLM-based Question Answering with Domain Hybrid Data](https://mlsanity.com/paper/2402.12869/) (2024)
- [The Manga Whisperer: Automatically Generating Transcriptions for Comics](https://mlsanity.com/paper/2401.10224/) (2024)
- [Manga109Dialog: A Large-scale Dialogue Dataset for Comics Speaker Detection](https://mlsanity.com/paper/2306.17469/) (2023)
- [A survey of comics research in computer science](https://mlsanity.com/paper/1804.05490/) (2018)
- [Unsupervised Manga Character Re-identification via Face-body and Spatial-temporal Associated Clustering](https://mlsanity.com/paper/2204.04621/) (2022)
- [Zero-Shot Character Identification and Speaker Prediction in Comics via Iterative Multimodal Fusion](https://mlsanity.com/paper/2404.13993/) (2024)
- [Identity-Aware Semi-Supervised Learning for Comic Character Re-Identification](https://mlsanity.com/paper/2308.09096/) (2023)
- [M2C: Towards Automatic Multimodal Manga Complement](https://mlsanity.com/paper/2310.17130/) (2023)
- [Comics for Everyone: Generating Accessible Text Descriptions for Comic Strips](https://mlsanity.com/paper/2310.00698/) (2023)
- [Multimodal Transformer for Comics Text-Cloze](https://mlsanity.com/paper/2403.03719/) (2024)
- [Visual onoma-to-wave: environmental sound synthesis from visual onomatopoeias and sound-source images](https://mlsanity.com/paper/2210.09173/) (2022)
- [MaRU: A Manga Retrieval and Understanding System Connecting Vision and Language](https://mlsanity.com/paper/2311.02083/) (2023)

## Built on by (most cited first, 2 of 2)

- [ComiCap: A VLMs pipeline for dense captioning of Comic Panels](https://mlsanity.com/paper/2409.16159/) (2024)
- [ComicScene154: A Scene Dataset for Comic Analysis](https://mlsanity.com/paper/2508.16190/) (2025)

## Abstract (arXiv)

This work explores how to fine-tune large language models using prompt engineering techniques with contextual information for generating an accurate text description of the full story, ready to be forwarded to off-the-shelve speech synthesis tools. We propose to use existing computer vision and optical character recognition techniques to build a grounded context from the comic strip image content, such as panels, characters, text, reading order and the association of bubbles and characters. Then we infer character identification and generate comic book script with context-aware panel description including character's appearance, posture, mood, dialogues etc. We believe that such enriched content description can be easily used to produce audiobook and eBook with various voices for characters, captions and playing sound effects.