# CoMix: A Comprehensive Benchmark for Multi-Task Comic Understanding

arXiv: https://arxiv.org/abs/2407.03550 · HTML: https://mlsanity.com/paper/2407.03550/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 42 references that are in the map.
- First posted on arXiv 4 Jul 2024.

## Builds on (most cited first, 14 of 14)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection](https://mlsanity.com/paper/2303.05499/) (2023)
- [Cross-Domain Weakly-Supervised Object Detection through Progressive Domain Adaptation](https://mlsanity.com/paper/1803.11365/) (2018)
- [Relationformer: A Unified Framework for Image-to-Graph Generation](https://mlsanity.com/paper/2203.10202/) (2022)
- [BAM! The Behance Artistic Media Dataset for Recognition Beyond Photography](https://mlsanity.com/paper/1704.08614/) (2017)
- [The Amazing Mysteries of the Gutter: Drawing Inferences Between Panels in Comic Book Narratives](https://mlsanity.com/paper/1611.05118/) (2016)
- [Towards Fully Automated Manga Translation](https://mlsanity.com/paper/2012.14271/) (2020)
- [The Manga Whisperer: Automatically Generating Transcriptions for Comics](https://mlsanity.com/paper/2401.10224/) (2024)
- [Object Detection for Comics using Manga109 Annotations](https://mlsanity.com/paper/1803.08670/) (2018)
- [Manga109Dialog: A Large-scale Dialogue Dataset for Comics Speaker Detection](https://mlsanity.com/paper/2306.17469/) (2023)
- [COO: Comic Onomatopoeia Dataset for Recognizing Arbitrary or Truncated Texts](https://mlsanity.com/paper/2207.04675/) (2022)
- [A survey of comics research in computer science](https://mlsanity.com/paper/1804.05490/) (2018)
- [Comics Datasets Framework: Mix of Comics datasets for detection benchmarking](https://mlsanity.com/paper/2407.03540/) (2024)
- [Multimodal Transformer for Comics Text-Cloze](https://mlsanity.com/paper/2403.03719/) (2024)

## Built on by (most cited first, 16 of 16)

- [MangaVQA and MangaLMM: A Benchmark and Specialized Model for Multimodal Manga Understanding](https://mlsanity.com/paper/2505.20298/) (2025)
- [ComicsPAP: understanding comic strips by picking the correct panel](https://mlsanity.com/paper/2503.08561/) (2025)
- [From Panels to Prose: Generating Literary Narratives from Comics](https://mlsanity.com/paper/2503.23344/) (2025)
- [Gaze Heads: How VLMs Look at What They Describe](https://mlsanity.com/paper/2606.14703/) (2026)
- [Beyond Single Frames: Can LMMs Comprehend Temporal and Contextual Narratives in Image Sequences?](https://mlsanity.com/paper/2502.13925/) (2025)
- [ComiCap: A VLMs pipeline for dense captioning of Comic Panels](https://mlsanity.com/paper/2409.16159/) (2024)
- [Re:Verse -- Can Your VLM Read a Manga?](https://mlsanity.com/paper/2508.08508/) (2025)
- [CoSMo: A Multimodal Transformer for Page Stream Segmentation in Comic Books](https://mlsanity.com/paper/2507.10053/) (2025)
- [Culture In a Frame: C3B as a Comic-Based Benchmark for Multimodal Culturally Awareness](https://mlsanity.com/paper/2510.00041/) (2025)
- [ComicScene154: A Scene Dataset for Comic Analysis](https://mlsanity.com/paper/2508.16190/) (2025)
- [Zooming into Comics: Region-Aware RL Improves Fine-Grained Comic Understanding in Vision-Language Models](https://mlsanity.com/paper/2511.06490/) (2025)
- [MangaFlow: An End-to-End Agentic Framework for Controllable Story to Manga Generation](https://mlsanity.com/paper/2605.28173/) (2026)
- [Semantic Similarity is a Spurious Measure of Comic Understanding: Lessons Learned from Hallucinations in a Benchmarking Experiment](https://mlsanity.com/paper/2603.01950/) (2026)
- [Question-Guided Evidence Acquisition for Multimodal Visual Question Answering](https://mlsanity.com/paper/2608.19739/) (2026)
- [ManGo: Manga Active Narrative Grounding Optimization](https://mlsanity.com/paper/2608.29865/) (2026)
- [Studying Comics at Scale with Artificial Intelligence: The PAGE Framework and Three Studies of American Comics and Japanese Manga](https://mlsanity.com/paper/2610.13840/) (2026)

## Abstract (arXiv)

The comic domain is rapidly advancing with the development of single-page analysis and synthesis models. However, evaluation metrics and datasets lag behind, often limited to small-scale or single-style test sets. We introduce a novel benchmark, CoMix, designed to evaluate the multi-task capabilities of models in comic analysis. Unlike existing benchmarks that focus on isolated tasks such as object detection or text recognition, CoMix addresses a broader range of tasks including object detection, speaker identification, character re-identification, reading order, and multi-modal reasoning tasks like character naming and dialogue generation. Our benchmark comprises three existing datasets with expanded annotations to support multi-task evaluation. To mitigate the over-representation of manga-style data, we have incorporated a new dataset of carefully selected American comic-style books, thereby enriching the diversity of comic styles. CoMix is designed to assess pre-trained models in zero-shot and limited fine-tuning settings, probing their transfer capabilities across different comic styles and tasks. The validation split of the benchmark is publicly available for research purposes, and an evaluation server for the held-out test split is also provided. Comparative results between human performance and state-of-the-art models reveal a significant performance gap, highlighting substantial opportunities for advancements in comic understanding. The dataset, baseline models, and code are accessible at https://github.com/emanuelevivoli/CoMix-dataset. This initiative sets a new standard for comprehensive comic analysis, providing the community with a common benchmark for evaluation on a large and varied set.