mlsanityData to 7 Oct 2026

ComicsPAP: understanding comic strips by picking the correct panel

arXiv 2503.08561 ↗First posted 11 Mar 2025cs.CVcited by 4

On the map: Multimodal LLMs / vision-language models → Image editing and quality assessment

Abstract (arXiv)

Large multimodal models (LMMs) have made impressive strides in image captioning, VQA, and video comprehension, yet they still struggle with the intricate temporal and spatial cues found in comics. To address this gap, we introduce ComicsPAP, a large-scale benchmark designed for comic strip understanding. Comprising over 100k samples and organized into 5 subtasks under a Pick-a-Panel framework, ComicsPAP demands models to identify the missing panel in a sequence. Our evaluations, conducted under both multi-image and single-image protocols, reveal that current state-of-the-art LMMs perform near chance on these tasks, underscoring significant limitations in capturing sequential and contextual dependencies. To close the gap, we adapted LMMs for comic strip understanding, obtaining better results on ComicsPAP than 10x bigger models, demonstrating that ComicsPAP offers a robust resource to drive future research in multimodal comic comprehension.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in Image editing and quality assessment, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 19 of its 27 references that are in the map. First posted on arXiv 11 Mar 2025.

Builds on

19 of 27 references.

References · 19

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  3. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,544
  4. 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suitescited by 1,033
  5. 2025VideoLLaMA 3: Frontier Multimodal Foundation Models for Image and Video Understandingcited by 502
  6. 2024Molmo and PixMo: Open Weights and Open Data for State-of-the-Art Vision-Language Modelscited by 321
  7. 2024MANTIS: Interleaved Multi-Image Instruction Tuningcited by 212
  8. 2024PaliGemma 2: A Family of Versatile VLMs for Transfercited by 204
  9. 2023Video Understanding with Large Language Models: A Surveycited by 203
  10. 2016The Amazing Mysteries of the Gutter: Drawing Inferences Between Panels in Comic Book Narrativescited by 41
  11. 2024The Manga Whisperer: Automatically Generating Transcriptions for Comicscited by 20
  12. 2024DiffSensei: Bridging Multi-Modal LLMs and Diffusion Models for Customized Manga Generationcited by 16
7 more
  1. 2024CoMix: A Comprehensive Benchmark for Multi-Task Comic Understandingcited by 15
  2. 2024Tails Tell Tales: Chapter-Wide Manga Transcriptions with Character Namescited by 10
  3. 2024One missing piece in Vision and Language: A Survey on Comics Understandingcited by 8
  4. 2024Comics Datasets Framework: Mix of Comics datasets for detection benchmarkingcited by 7
  5. 2024MangaUB: A Manga Understanding Benchmark for Large Multimodal Modelscited by 7
  6. 2024Multimodal Transformer for Comics Text-Clozecited by 5
  7. 2024ComiCap: A VLMs pipeline for dense captioning of Comic Panelscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 2.

4 papers.

  1. 2025Zooming into Comics: Region-Aware RL Improves Fine-Grained Comic Understanding in Vision-Language Modelscited by 1
  2. 2025DreamingComics: A Story Visualization Pipeline via Subject and Layout Customized Generation using Video Modelscited by 1
  3. 2026Computational Humor with Multimodal LLMs: Methods, Datasets, Evaluation, and Challengescited by 0
  4. 2026ManGo: Manga Active Narrative Grounding Optimizationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Humor in Pixels: Benchmarking Large Multimodal Models Understanding of Online Comics
  2. 2023Comics for Everyone: Generating Accessible Text Descriptions for Comic Strips
  3. 2025CoSMo: A Multimodal Transformer for Page Stream Segmentation in Comic Books
  4. 2026Thinking with Comics: Enhancing Multimodal Reasoning through Structured Visual Storytelling
  5. 2026Towards Faithful Reasoning in Comics for Small MLLMs
  6. 2025Beyond Single Frames: Can LMMs Comprehend Temporal and Contextual Narratives in Image Sequences?
  7. 2025MangaVQA and MangaLMM: A Benchmark and Specialized Model for Multimodal Manga Understanding
  8. 2025From Panels to Prose: Generating Literary Narratives from Comics