# Re:Verse -- Can Your VLM Read a Manga?

arXiv: https://arxiv.org/abs/2508.08508 · HTML: https://mlsanity.com/paper/2508.08508/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 24 of its 36 references that are in the map.
- First posted on arXiv 11 Aug 2025.

## Placed in

- Multimodal LLMs / vision-language models / [MLLM training and visual instruction tuning](https://mlsanity.com/mllm/mllm-training-and-visual-instruction-tuning/)
- Retrieval-augmented generation / [Multimodal RAG](https://mlsanity.com/rag/multimodal-rag/)

## Builds on (most cited first, 24 of 24)

- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation](https://mlsanity.com/paper/2201.12086/) (2022)
- [MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models](https://mlsanity.com/paper/2304.10592/) (2023)
- [Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond](https://mlsanity.com/paper/2308.12966/) (2023)
- [Vision-Language Models for Vision Tasks: A Survey](https://mlsanity.com/paper/2304.00685/) (2023)
- [A Survey on Benchmarks of Multimodal Large Language Models](https://mlsanity.com/paper/2408.08632/) (2024)
- [Of Human Criteria and Automatic Metrics: A Benchmark of the Evaluation of Story Generation](https://mlsanity.com/paper/2208.11646/) (2022)
- [TOMATO: Assessing Visual Temporal Reasoning Capabilities in Multimodal Foundation Models](https://mlsanity.com/paper/2410.23266/) (2024)
- [OpenMEVA: A Benchmark for Evaluating Open-ended Story Generation Metrics](https://mlsanity.com/paper/2105.08920/) (2021)
- [VHELM: A Holistic Evaluation of Vision Language Models](https://mlsanity.com/paper/2410.07112/) (2024)
- [The Amazing Mysteries of the Gutter: Drawing Inferences Between Panels in Comic Book Narratives](https://mlsanity.com/paper/1611.05118/) (2016)
- [SCORE: Story Coherence and Retrieval Enhancement for AI Narratives](https://mlsanity.com/paper/2503.23512/) (2025)
- [Building a Manga Dataset "Manga109" with Annotations for Multimedia Applications](https://mlsanity.com/paper/2005.04425/) (2020)
- [Do Vision and Language Models Share Concepts? A Vector Space Alignment Study](https://mlsanity.com/paper/2302.06555/) (2023)
- [What Makes a Good Story and How Can We Measure It? A Comprehensive Survey of Story Evaluation](https://mlsanity.com/paper/2408.14622/) (2024)
- [CoMix: A Comprehensive Benchmark for Multi-Task Comic Understanding](https://mlsanity.com/paper/2407.03550/) (2024)
- [Transferring Pre-trained Multimodal Representations with Cross-modal Similarity Matching](https://mlsanity.com/paper/2301.02903/) (2023)
- [MangaUB: A Manga Understanding Benchmark for Large Multimodal Models](https://mlsanity.com/paper/2407.19034/) (2024)
- [Fabula Entropy Indexing: Objective Measures of Story Coherence](https://mlsanity.com/paper/2104.07472/) (2021)
- [M2C: Towards Automatic Multimodal Manga Complement](https://mlsanity.com/paper/2310.17130/) (2023)
- [From Panels to Prose: Generating Literary Narratives from Comics](https://mlsanity.com/paper/2503.23344/) (2025)
- [A Comprehensive Gold Standard and Benchmark for Comics Text Detection and Recognition](https://mlsanity.com/paper/2212.14674/) (2022)
- [ImageChain: Advancing Sequential Image-to-Text Reasoning in Multimodal Large Language Models](https://mlsanity.com/paper/2502.19409/) (2025)
- [MaRU: A Manga Retrieval and Understanding System Connecting Vision and Language](https://mlsanity.com/paper/2311.02083/) (2023)

## Built on by (most cited first, 3 of 3)

- [ExpressEdit: Fast Editing of Stylized Facial Expressions with Diffusion Models in Photoshop](https://mlsanity.com/paper/2604.03448/) (2026)
- [BareBones: Benchmarking Zero-Shot Geometric Comprehension in VLMs](https://mlsanity.com/paper/2604.10528/) (2026)
- [Re:Cognize -- Open-Set Comic Character Re-Identification](https://mlsanity.com/paper/2609.34032/) (2026)

## Abstract (arXiv)

Current Vision Language Models (VLMs) demonstrate a critical gap between surface-level recognition and deep narrative reasoning when processing sequential visual storytelling. Through a comprehensive investigation of manga narrative understanding, we reveal that while recent large multimodal models excel at individual panel interpretation, they systematically fail at temporal causality and cross-panel cohesion, core requirements for coherent story comprehension. We introduce a novel evaluation framework that combines fine-grained multimodal annotation, cross-modal embedding analysis, and retrieval-augmented assessment to systematically characterize these limitations. Our methodology includes (i) a rigorous annotation protocol linking visual elements to narrative structure through aligned light novel text, (ii) comprehensive evaluation across multiple reasoning paradigms, including direct inference and retrieval-augmented generation, and (iii) cross-modal similarity analysis revealing fundamental misalignments in current VLMs' joint representations. Applying this framework to Re:Zero manga across 11 chapters with 308 annotated panels, we conduct the first systematic study of long-form narrative understanding in VLMs through three core evaluation axes: generative storytelling, contextual dialogue grounding, and temporal reasoning. Our findings demonstrate that current models lack genuine story-level intelligence, struggling particularly with non-linear narratives, character consistency, and causal inference across extended sequences. This work establishes both the foundation and practical methodology for evaluating narrative intelligence, while providing actionable insights into the capability of deep sequential understanding of Discrete Visual Narratives beyond basic recognition in Multimodal Models. Project Page: https://re-verse.vercel.app