mlsanityData to 7 Oct 2026

Summarization of Multimodal Presentations with Vision-Language Models: Study of the Effect of Modalities and Structure

arXiv 2504.10049 ↗First posted 14 Apr 2025cs.CV cs.CLcited by 0

On the map: Multimodal LLMs / vision-language models → Video understanding with MLLMs

Abstract (arXiv)

Vision-Language Models (VLMs) can process visual and textual information in multiple formats: texts, images, interleaved texts and images, or even hour-long videos. In this work, we conduct fine-grained quantitative and qualitative analyses of automatic summarization of multimodal presentations using VLMs with various representations as input. From these experiments, we suggest cost-effective strategies for generating summaries from text-heavy multimodal documents under different input-length budgets using VLMs. We show that slides extracted from the video stream can be beneficially used as input against the raw video, and that a structured representation from interleaved slides and transcript provides the best performance. Finally, we reflect and comment on the nature of cross-modal interactions in multimodal presentations and share suggestions to improve the capabilities of VLMs to understand documents of this nature.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Video understanding with MLLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 31 of its 50 references that are in the map. First posted on arXiv 14 Apr 2025.

Builds on

31 of 50 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understanding22 Jul
  2. 2024Long Context Transfer from Language to Vision24 Jun
  3. 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projection16 Nov

Other references · 28

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  5. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  6. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  7. 2020Longformer: The Long-Document Transformercited by 3,362
  8. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,048
  9. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,551
  10. 2020Big Bird: Transformers for Longer Sequencescited by 1,673
  11. 2021BARTScore: Evaluating Generated Text as Text Generationcited by 719
  12. 2022Mind the Gap: Understanding the Modality Gap in Multi-modal Contrastive Representation Learningcited by 553
16 more
  1. 2025SmolLM2: When Smol Goes Big -- Data-Centric Training of a Small Language Modelcited by 307
  2. 2024Long-context LLMs Struggle with Long In-context Learningcited by 285
  3. 2021LongT5: Efficient Text-To-Text Transformer for Long Sequencescited by 232
  4. 2023Valley: Video Assistant with Large Language model Enhanced abilitYcited by 231
  5. 2023Can Pre-trained Vision and Language Models Answer Visual Information-Seeking Questions?cited by 203
  6. 2024General OCR Theory: Towards OCR-2.0 via a Unified End-to-end Modelcited by 174
  7. 2019Towards Multimodal Sarcasm Detection (An _Obviously_ Perfect Paper)cited by 173
  8. 2025Words or Vision: Do Vision-Language Models Have Blind Faith in Text?cited by 86
  9. 2021Video Summarization Using Deep Neural Networks: A Surveycited by 85
  10. 2024A Systematic Survey of Text Summarization: From Statistical Methods to Large Language Modelscited by 52
  11. 2020GRUEN for Evaluating Linguistic Quality of Generated Textcited by 37
  12. 2018Learn to Combine Modalities in Multimodal Deep Learningcited by 29
  13. 2022How Far are We from Robust Long Abstractive Summarization?cited by 25
  14. 2025What Is That Talk About? A Video-to-Text Summarization Dataset for Scientific Presentationscited by 11
  15. 2021Estimation of Summary-to-Text Inconsistency by Mismatched Embeddingscited by 5
  16. 2024Mitigating the Impact of Reference Quality on Evaluation of Summarization Systems with Reference-Free Metricscited by 5

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Integrating Video and Text: A Balanced Approach to Multimodal Summary Generation and Evaluation
  2. 2025PosterSum: A Multimodal Benchmark for Scientific Poster Summarization
  3. 2026Multimodal Abstractive Summarization of Instructional Videos with Vision-Language Models
  4. 2021UniMS: A Unified Framework for Multimodal Summarization with Knowledge Distillation
  5. 2025M-DocSum: Do LVLMs Genuinely Comprehend Interleaved Image-Text in Document Summarization?
  6. 2026Measuring What Matters Beyond Text: Evaluating Multimodal Summaries by Quality, Alignment, and Diversity
  7. 2026MMLDSum-LLM: Multimodal Long-Document Summarization with Visual-Alignment and Keyword-Aware
  8. 2025VLM-SlideEval: Evaluating VLMs on Structured Comprehension and Perturbation Sensitivity in PPT