mlsanityData to 14 Oct 2026

Re:Verse -- Can Your VLM Read a Manga?

arXiv 2508.08508 ↗First posted 11 Aug 2025cs.CV cs.CLcited by 3

On the map: Multimodal LLMs / vision-language models → MLLM training and visual instruction tuning

also in 1 more
Abstract (arXiv)

Current Vision Language Models (VLMs) demonstrate a critical gap between surface-level recognition and deep narrative reasoning when processing sequential visual storytelling. Through a comprehensive investigation of manga narrative understanding, we reveal that while recent large multimodal models excel at individual panel interpretation, they systematically fail at temporal causality and cross-panel cohesion, core requirements for coherent story comprehension. We introduce a novel evaluation framework that combines fine-grained multimodal annotation, cross-modal embedding analysis, and retrieval-augmented assessment to systematically characterize these limitations. Our methodology includes (i) a rigorous annotation protocol linking visual elements to narrative structure through aligned light novel text, (ii) comprehensive evaluation across multiple reasoning paradigms, including direct inference and retrieval-augmented generation, and (iii) cross-modal similarity analysis revealing fundamental misalignments in current VLMs' joint representations. Applying this framework to Re:Zero manga across 11 chapters with 308 annotated panels, we conduct the first systematic study of long-form narrative understanding in VLMs through three core evaluation axes: generative storytelling, contextual dialogue grounding, and temporal reasoning. Our findings demonstrate that current models lack genuine story-level intelligence, struggling particularly with non-linear narratives, character consistency, and causal inference across extended sequences. This work establishes both the foundation and practical methodology for evaluating narrative intelligence, while providing actionable insights into the capability of deep sequential understanding of Discrete Visual Narratives beyond basic recognition in Multimodal Models. Project Page: https://re-verse.vercel.app

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 24 of its 36 references that are in the map. First posted on arXiv 11 Aug 2025.

Builds on

24 of 36 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025From Panels to Prose: Generating Literary Narratives from Comics30 Mar
  2. 2025ImageChain: Advancing Sequential Image-to-Text Reasoning in Multimodal Large Language Models26 Feb
  3. 2024A Survey on Benchmarks of Multimodal Large Language Models16 Aug
  4. 2024MangaUB: A Manga Understanding Benchmark for Large Multimodal Models26 Jul

Other references · 20

  1. 2023Visual Instruction Tuningcited by 7,869
  2. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
  3. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,711
  4. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,224
  5. 2023Vision-Language Models for Vision Tasks: A Surveycited by 839
  6. 2022Of Human Criteria and Automatic Metrics: A Benchmark of the Evaluation of Story Generationcited by 75
  7. 2024TOMATO: Assessing Visual Temporal Reasoning Capabilities in Multimodal Foundation Modelscited by 73
  8. 2021OpenMEVA: A Benchmark for Evaluating Open-ended Story Generation Metricscited by 61
  9. 2024VHELM: A Holistic Evaluation of Vision Language Modelscited by 53
  10. 2016The Amazing Mysteries of the Gutter: Drawing Inferences Between Panels in Comic Book Narrativescited by 42
  11. 2025SCORE: Story Coherence and Retrieval Enhancement for AI Narrativescited by 42
  12. 2020Building a Manga Dataset "Manga109" with Annotations for Multimedia Applicationscited by 38
8 more
  1. 2023Do Vision and Language Models Share Concepts? A Vector Space Alignment Studycited by 27
  2. 2024What Makes a Good Story and How Can We Measure It? A Comprehensive Survey of Story Evaluationcited by 17
  3. 2024CoMix: A Comprehensive Benchmark for Multi-Task Comic Understandingcited by 16
  4. 2023Transferring Pre-trained Multimodal Representations with Cross-modal Similarity Matchingcited by 14
  5. 2021Fabula Entropy Indexing: Objective Measures of Story Coherencecited by 7
  6. 2023M2C: Towards Automatic Multimodal Manga Complementcited by 7
  7. 2022A Comprehensive Gold Standard and Benchmark for Comics Text Detection and Recognitioncited by 4
  8. 2023MaRU: A Manga Retrieval and Understanding System Connecting Vision and Languagecited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025MangaVQA and MangaLMM: A Benchmark and Specialized Model for Multimodal Manga Understanding
  2. 2025Hierarchical Knowledge Graphs for Story Understanding in Visual Narratives
  3. 2025Zooming into Comics: Region-Aware RL Improves Fine-Grained Comic Understanding in Vision-Language Models
  4. 2025Robust Symbolic Reasoning for Visual Narratives via Hierarchical and Semantically Normalized Knowledge Graphs
  5. 2026Text-Only vs. Image-Aware VLM Judges for Manga Translation Evaluation
  6. 2025ReadBench: Measuring the Dense Text Visual Reading Ability of Vision-Language Models
  7. 2026NarrativeTrack: Evaluating Entity-Centric Reasoning for Narrative Understanding
  8. 2023TouchStone: Evaluating Vision-Language Models by Language Models