mlsanityData to 7 Oct 2026

PosterSum: A Multimodal Benchmark for Scientific Poster Summarization

arXiv 2502.17540 ↗First posted 24 Feb 2025cs.CV cs.AI cs.CLcited by 10

On the map: Multimodal LLMs / vision-language models → Chart understanding and anomaly detection

Abstract (arXiv)

Generating accurate and concise textual summaries from multimodal documents is challenging, especially when dealing with visually complex content like scientific posters. We introduce PosterSum, a novel benchmark to advance the development of vision-language models that can understand and summarize scientific posters into research paper abstracts. Our dataset contains 16,305 conference posters paired with their corresponding abstracts as summaries. Each poster is provided in image format and presents diverse visual understanding challenges, such as complex layouts, dense text regions, tables, and figures. We benchmark state-of-the-art Multimodal Large Language Models (MLLMs) on PosterSum and demonstrate that they struggle to accurately interpret and summarize scientific posters. We propose Segment & Summarize, a hierarchical method that outperforms current MLLMs on automated metrics, achieving a 3.14% gain in ROUGE-L. This will serve as a starting point for future research on poster summarization.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Placed in Chart understanding and anomaly detection, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 32 of its 53 references that are in the map. First posted on arXiv 24 Feb 2025.

Builds on

32 of 53 references.

References · 32

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  3. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  4. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,388
  5. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,105
  6. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,044
  7. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,790
  8. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,582
  9. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,449
  10. 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,259
  11. 2021SummaC: Re-Visiting NLI-based Models for Inconsistency Detection in Summarizationcited by 402
  12. 2021QAFactEval: Improved QA-Based Factual Consistency Evaluation for Summarizationcited by 233
20 more
  1. 2023SlideVQA: A Dataset for Document Visual Question Answering on Multiple Imagescited by 175
  2. 2020TLDR: Extreme Summarization of Scientific Documentscited by 136
  3. 2024mPLUG-DocOwl2: High-resolution Compressing for OCR-free Multi-page Document Understandingcited by 113
  4. 2023DocLLM: A layout-aware generative language model for multimodal document understandingcited by 112
  5. 2024LayoutLLM: Layout Instruction Tuning with Large Language Models for Document Understandingcited by 94
  6. 2023CapsFusion: Rethinking Image-Text Data at Scalecited by 90
  7. 2024SPIQA: A Dataset for Multimodal Question Answering on Scientific Paperscited by 75
  8. 2019ScisummNet: A Large Annotated Corpus and Content-Impact Models for Scientific Paper Summarization with Citation Networkscited by 73
  9. 2022ERNIE-Layout: Layout Knowledge Enhanced Pre-training for Visually-rich Document Understandingcited by 67
  10. 2023DocFormerv2: Local Features for Document Understandingcited by 46
  11. 2021MMOCR: A Comprehensive Toolbox for Text Detection, Recognition and Understandingcited by 40
  12. 2019TalkSumm: A Dataset and Scalable Annotation Method for Scientific Paper Summarization Based on Conference Talkscited by 20
  13. 2022Mix and Reason: Reasoning over Semantic Topology with Data Mixing for Domain Generalizationcited by 20
  14. 2024ACLSum: A New Dataset for Aspect-based Summarization of Scientific Publicationscited by 20
  15. 2021Leveraging Information Bottleneck for Scientific Document Summarizationcited by 13
  16. 2024SciNews: From Scholarly Complexities to Public Narratives -- A Dataset for Scientific News Report Generationcited by 12
  17. 2024Select and Summarize: Scene Saliency for Movie Script Summarizationcited by 9
  18. 2022TSTR: Too Short to Represent, Summarize with Details! Intro-Guided Extended Summary Generationcited by 4
  19. 2023Divide and Conquer: Answering Questions with Object Factorization and Compositional Reasoningcited by 4
  20. 2022Addressing Mistake Severity in Neural Networks with Semantic Knowledgecited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 8; 2026: 2.

10 papers.

  1. 2025Paper2Poster: Towards Multimodal Poster Automation from Scientific Paperscited by 56
  2. 2025P2P: Automated Paper-to-Poster Generation and Fine-Grained Benchmarkcited by 15
  3. 2025PosterGen: Aesthetic-Aware Multi-Modal Paper-to-Poster Generation via Multi-Agent LLMscited by 8
  4. 2025PosterForest: Hierarchical Multi-Agent Collaboration for Scientific Poster Generationcited by 7
  5. 2026Any2Poster: Any-Source Poster Generation Across Modalities and Domainscited by 5
  6. 2025PRISMM-Bench: A Benchmark of Peer-Review Grounded Multimodal Inconsistenciescited by 4
  7. 2025SciPostGen: Bridging the Gap between Scientific Papers and Poster Layoutscited by 4
  8. 2025Training Multi-Image Vision Agents via End2End Reinforcement Learningcited by 4
  9. 2025VideoAgent: Personalized Synthesis of Scientific Videoscited by 2
  10. 2026TextRich: A Multi-Domain Benchmark for Detecting AI-Generated Text-Rich Images from GPT-Image-2cited by 2

Similar, not cited

Close in wording, no citation either way

  1. 2025Summarization of Multimodal Presentations with Vision-Language Models: Study of the Effect of Modalities and Structure
  2. 2025MAC: A Live Benchmark for Multimodal Large Language Models in Scientific Understanding
  3. 2026Unifying Scientific Communication: Fine-Grained Correspondence Across Scientific Media
  4. 2026Benchmarking Multimodal Large Language Models for Scientific Visualization Literacy
  5. 2026Measuring What Matters Beyond Text: Evaluating Multimodal Summaries by Quality, Alignment, and Diversity
  6. 2021UniMS: A Unified Framework for Multimodal Summarization with Knowledge Distillation
  7. 2025M-DocSum: Do LVLMs Genuinely Comprehend Interleaved Image-Text in Document Summarization?
  8. 2026EfficientPosterGen: Semantic-aware Efficient Poster Generation via Token Compression and Accurate Violation Detection