# MangaVQA and MangaLMM: A Benchmark and Specialized Model for Multimodal Manga Understanding

arXiv: https://arxiv.org/abs/2505.20298 · HTML: https://mlsanity.com/paper/2505.20298/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 45 references that are in the map.
- First posted on arXiv 26 May 2025.

## Builds on (most cited first, 30 of 35)

- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024)
- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [LLaVA-OneVision: Easy Visual Task Transfer](https://mlsanity.com/paper/2408.03326/) (2024)
- [MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI](https://mlsanity.com/paper/2311.16502/) (2023)
- [Towards VQA Models That Can Read](https://mlsanity.com/paper/1904.08920/) (2019)
- [DocVQA: A Dataset for VQA on Document Images](https://mlsanity.com/paper/2007.00398/) (2020)
- [Gradient Surgery for Multi-Task Learning](https://mlsanity.com/paper/2001.06782/) (2020)
- [Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAs](https://mlsanity.com/paper/2503.01743/) (2025)
- [Sketch-based Manga Retrieval using Manga109 Dataset](https://mlsanity.com/paper/1510.04389/) (2015)
- [TextMonkey: An OCR-Free Large Multimodal Model for Understanding Document](https://mlsanity.com/paper/2403.04473/) (2024)
- [Attentive Single-Tasking of Multiple Tasks](https://mlsanity.com/paper/1904.08918/) (2019)
- [ICDAR2019 Robust Reading Challenge on Arbitrary-Shaped Text (RRC-ArT)](https://mlsanity.com/paper/1909.07145/) (2019)
- [ICDAR2019 Robust Reading Challenge on Multi-lingual Scene Text Detection and Recognition -- RRC-MLT-2019](https://mlsanity.com/paper/1907.00945/) (2019)
- [ICDAR 2019 Robust Reading Challenge on Reading Chinese Text on Signboard](https://mlsanity.com/paper/1912.09641/) (2019)
- [ICDAR 2019 Competition on Large-scale Street View Text with Partial Labeling -- RRC-LSVT](https://mlsanity.com/paper/1909.07741/) (2019)
- [DeepSolo: Let Transformer Decoder with Explicit Points Solo for Text Spotting](https://mlsanity.com/paper/2211.10772/) (2022)
- [Pangea: A Fully Open Multilingual Multimodal LLM for 39 Languages](https://mlsanity.com/paper/2410.16153/) (2024)
- [DetToolChain: A New Prompting Paradigm to Unleash Detection Ability of MLLM](https://mlsanity.com/paper/2403.12488/) (2024)
- [The Amazing Mysteries of the Gutter: Drawing Inferences Between Panels in Comic Book Narratives](https://mlsanity.com/paper/1611.05118/) (2016)
- [Building a Manga Dataset "Manga109" with Annotations for Multimedia Applications](https://mlsanity.com/paper/2005.04425/) (2020)
- [PALO: A Polyglot Large Multimodal Model for 5B People](https://mlsanity.com/paper/2402.14818/) (2024)
- [ESTextSpotter: Towards Better Scene Text Spotting with Explicit Synergy in Transformer](https://mlsanity.com/paper/2308.10147/) (2023)
- [Octavius: Mitigating Task Interference in MLLMs via LoRA-MoE](https://mlsanity.com/paper/2311.02684/) (2023)
- [The Manga Whisperer: Automatically Generating Transcriptions for Comics](https://mlsanity.com/paper/2401.10224/) (2024)
- [Mitigating Task Interference in Multi-Task Learning via Explicit Task Routing with Non-Learnable Primitives](https://mlsanity.com/paper/2308.02066/) (2023)
- [CoMix: A Comprehensive Benchmark for Multi-Task Comic Understanding](https://mlsanity.com/paper/2407.03550/) (2024)
- [COO: Comic Onomatopoeia Dataset for Recognizing Arbitrary or Truncated Texts](https://mlsanity.com/paper/2207.04675/) (2022)

## Built on by (most cited first, 7 of 7)

- [Evaluating Multimodal Large Language Models on Vertically Written Japanese Text](https://mlsanity.com/paper/2511.15059/) (2025)
- [Zooming into Comics: Region-Aware RL Improves Fine-Grained Comic Understanding in Vision-Language Models](https://mlsanity.com/paper/2511.06490/) (2025)
- [Manga109-v2026: Revisiting Manga109 Annotations for Modern Manga Understanding](https://mlsanity.com/paper/2605.21182/) (2026)
- [JMMMU-Pro: Image-based Japanese Multi-discipline Multimodal Understanding Benchmark via Vibe Benchmark Construction](https://mlsanity.com/paper/2512.14620/) (2025)
- [JaWildText: A Benchmark for Vision-Language Models on Japanese Scene Text Understanding](https://mlsanity.com/paper/2603.27942/) (2026)
- [ManGo: Manga Active Narrative Grounding Optimization](https://mlsanity.com/paper/2608.29865/) (2026)
- [Studying Comics at Scale with Artificial Intelligence: The PAGE Framework and Three Studies of American Comics and Japanese Manga](https://mlsanity.com/paper/2610.13840/) (2026)

## Abstract (arXiv)

Manga, or Japanese comics, is a richly multimodal narrative form that blends images and text in complex ways. Teaching large multimodal models (LMMs) to understand such narratives at a human-like level could help manga creators reflect on and refine their stories. To this end, we introduce two benchmarks for multimodal manga understanding: MangaOCR, which targets in-page text recognition, and MangaVQA, a novel benchmark designed to evaluate contextual understanding through visual question answering. MangaVQA consists of 526 high-quality, manually constructed question-answer pairs, enabling reliable evaluation across diverse narrative and visual scenarios. Building on these benchmarks, we develop MangaLMM, a manga-specialized model finetuned from the open-source LMM Qwen2.5-VL to jointly handle both tasks. Through extensive experiments, including comparisons with proprietary models such as GPT-4o and Gemini 2.5, we assess how well LMMs understand manga. Our benchmark and model provide a comprehensive foundation for evaluating and advancing LMMs in the richly narrative domain of manga.