# MangaUB: A Manga Understanding Benchmark for Large Multimodal Models

arXiv: https://arxiv.org/abs/2407.19034 · HTML: https://mlsanity.com/paper/2407.19034/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 15 of its 23 references that are in the map.
- First posted on arXiv 26 Jul 2024.

## Placed in

- Multimodal LLMs / vision-language models / [MLLM training and visual instruction tuning](https://mlsanity.com/mllm/mllm-training-and-visual-instruction-tuning/)

## Builds on (most cited first, 15 of 15)

- [Improved Baselines with Visual Instruction Tuning](https://mlsanity.com/paper/2310.03744/) (2023)
- [Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond](https://mlsanity.com/paper/2308.12966/) (2023)
- [MMBench: Is Your Multi-modal Model an All-around Player?](https://mlsanity.com/paper/2307.06281/) (2023)
- [DocVQA: A Dataset for VQA on Document Images](https://mlsanity.com/paper/2007.00398/) (2020)
- [CogVLM: Visual Expert for Pretrained Language Models](https://mlsanity.com/paper/2311.03079/) (2023)
- [Sketch-based Manga Retrieval using Manga109 Dataset](https://mlsanity.com/paper/1510.04389/) (2015)
- [Evolutionary Optimization of Model Merging Recipes](https://mlsanity.com/paper/2403.13187/) (2024)
- [ScreenAI: A Vision-Language Model for UI and Infographics Understanding](https://mlsanity.com/paper/2402.04615/) (2024)
- [The Amazing Mysteries of the Gutter: Drawing Inferences Between Panels in Comic Book Narratives](https://mlsanity.com/paper/1611.05118/) (2016)
- [Building a Manga Dataset "Manga109" with Annotations for Multimedia Applications](https://mlsanity.com/paper/2005.04425/) (2020)
- [Towards Fully Automated Manga Translation](https://mlsanity.com/paper/2012.14271/) (2020)
- [The Manga Whisperer: Automatically Generating Transcriptions for Comics](https://mlsanity.com/paper/2401.10224/) (2024)
- [COO: Comic Onomatopoeia Dataset for Recognizing Arbitrary or Truncated Texts](https://mlsanity.com/paper/2207.04675/) (2022)
- [A survey of comics research in computer science](https://mlsanity.com/paper/1804.05490/) (2018)
- [Comics Datasets Framework: Mix of Comics datasets for detection benchmarking](https://mlsanity.com/paper/2407.03540/) (2024)

## Built on by (most cited first, 8 of 8)

- [MangaVQA and MangaLMM: A Benchmark and Specialized Model for Multimodal Manga Understanding](https://mlsanity.com/paper/2505.20298/) (2025)
- [ComicsPAP: understanding comic strips by picking the correct panel](https://mlsanity.com/paper/2503.08561/) (2025)
- [Re:Verse -- Can Your VLM Read a Manga?](https://mlsanity.com/paper/2508.08508/) (2025)
- [Zooming into Comics: Region-Aware RL Improves Fine-Grained Comic Understanding in Vision-Language Models](https://mlsanity.com/paper/2511.06490/) (2025)
- [Manga109-v2026: Revisiting Manga109 Annotations for Modern Manga Understanding](https://mlsanity.com/paper/2605.21182/) (2026)
- [Understanding Multi-Agent Reasoning with Large Language Models for Cartoon VQA](https://mlsanity.com/paper/2601.03073/) (2026)
- [Computational Humor with Multimodal LLMs: Methods, Datasets, Evaluation, and Challenges](https://mlsanity.com/paper/2607.19011/) (2026)
- [Studying Comics at Scale with Artificial Intelligence: The PAGE Framework and Three Studies of American Comics and Japanese Manga](https://mlsanity.com/paper/2610.13840/) (2026)

## Abstract (arXiv)

Manga is a popular medium that combines stylized drawings and text to convey stories. As manga panels differ from natural images, computational systems traditionally had to be designed specifically for manga. Recently, the adaptive nature of modern large multimodal models (LMMs) shows possibilities for more general approaches. To provide an analysis of the current capability of LMMs for manga understanding tasks and identifying areas for their improvement, we design and evaluate MangaUB, a novel manga understanding benchmark for LMMs. MangaUB is designed to assess the recognition and understanding of content shown in a single panel as well as conveyed across multiple panels, allowing for a fine-grained analysis of a model's various capabilities required for manga understanding. Our results show strong performance on the recognition of image content, while understanding the emotion and information conveyed across multiple panels is still challenging, highlighting future work towards LMMs for manga understanding.