mlsanityData to 7 Oct 2026

What Is That Talk About? A Video-to-Text Summarization Dataset for Scientific Presentations

arXiv 2502.08279 ↗First posted 12 Feb 2025cs.CL cs.AI cs.CVcited by 11

Not on the map. Why?

Abstract (arXiv)

Transforming recorded videos into concise and accurate textual summaries is a growing challenge in multimodal learning. This paper introduces VISTA, a dataset specifically designed for video-to-text summarization in scientific domains. VISTA contains 18,599 recorded AI conference presentations paired with their corresponding paper abstracts. We benchmark the performance of state-of-the-art large models and apply a plan-based framework to better capture the structured nature of abstracts. Both human and automated evaluations confirm that explicit planning enhances summary quality and factual consistency. However, a considerable gap remains between models and human performance, highlighting the challenges of our dataset. This study aims to pave the way for future research on scientific video-to-text summarization.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 61 of its 97 references that are in the map. First posted on arXiv 12 Feb 2025.

Builds on

61 of 97 references.

References · 61

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  4. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  5. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  6. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,044
  7. 2014CIDEr: Consensus-based Image Description Evaluationcited by 2,027
  8. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,790
  9. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,582
  10. 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understandingcited by 1,461
  11. 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,385
  12. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,103
38 more
  1. 2015A Diversity-Promoting Objective Function for Neural Conversation Modelscited by 1,068
  2. 2024BLINK: Multimodal Large Language Models Can See but Not Perceivecited by 529
  3. 2024Hallucination of Multimodal Large Language Models: A Surveycited by 380
  4. 2024MLLM-as-a-Judge: Assessing Multimodal LLM-as-a-Judge with Vision-Language Benchmarkcited by 362
  5. 2024VideoScore: Building Automatic Metrics to Simulate Fine-grained Human Feedback for Video Generationcited by 190
  6. 2024Aligning with Human Judgement: The Role of Pairwise Preference in Large Language Model Evaluatorscited by 146
  7. 2018How2: A Large-scale Dataset for Multimodal Language Understandingcited by 137
  8. 2020TLDR: Extreme Summarization of Scientific Documentscited by 136
  9. 2023Grounding Language Models to Images for Multimodal Inputs and Outputscited by 129
  10. 2024A Comprehensive Survey of Scientific Large Language Models and Their Applications in Scientific Discoverycited by 96
  11. 2024Video ReCap: Recursive Captioning of Hour-Long Videoscited by 88
  12. 2021Planning with Learned Entity Prompts for Abstractive Summarizationcited by 75
  13. 2024SPIQA: A Dataset for Multimodal Question Answering on Scientific Paperscited by 75
  14. 2023Align and Attend: Multimodal Summarization with Dual Contrastive Lossescited by 65
  15. 2023Shot2Story: A New Benchmark for Comprehensive Understanding of Multi-shot Videoscited by 54
  16. 2024From Pixels to Insights: A Survey on Automatic Chart Understanding in the Era of Large Foundation Modelscited by 52
  17. 2023ChatGPT vs Human-authored Text: Insights into Controllable Text Summarization and Sentence Style Transfercited by 49
  18. 2024V2Xum-LLM: Cross-Modal Video Summarization with Temporal Prompt Instruction Tuningcited by 45
  19. 2020VMSMO: Learning to Generate Multimodal Summary for Video-based News Articlescited by 44
  20. 2022Conditional Generation with a Question-Answering Blueprintcited by 44
  21. 2023Can We Edit Multimodal Large Language Models?cited by 44
  22. 2023VideoXum: Cross-modal Visual and Textural Summarization of Videoscited by 41
  23. 2021When Does Translation Require Context? A Data-driven, Multilingual Explorationcited by 36
  24. 2021Controllable Neural Dialogue Summarization with Personal Named Entity Planningcited by 36
  25. 2024Tuning Large Multimodal Models for Videos using Reinforcement Learning from AI Feedbackcited by 36
  26. 2020Unsupervised Opinion Summarization with Content Planningcited by 27
  27. 2018Video Storytelling: Textual Summaries for Eventscited by 22
  28. 2019TalkSumm: A Dataset and Scalable Annotation Method for Scientific Paper Summarization Based on Conference Talkscited by 20
  29. 2024ACLSum: A New Dataset for Aspect-based Summarization of Scientific Publicationscited by 20
  30. 2020Learning Kernel Tests Without Data Splittingcited by 19
  31. 2023Models See Hallucinations: Evaluating the Factuality in Video Captioningcited by 19
  32. 2023QUDEVAL: The Evaluation of Questions Under Discussion Discourse Parsingcited by 19
  33. 2023Incorporating Distributions of Discourse Structure for Long Document Abstractive Summarizationcited by 17
  34. 2021StreamHover: Livestream Transcript Summarization and Annotationcited by 15
  35. 2023Elaborative Simplification as Implicit Questions Under Discussioncited by 14
  36. 2021Leveraging Information Bottleneck for Scientific Document Summarizationcited by 13
  37. 2024A Modular Approach for Multimodal Summarization of TV Showscited by 13
  38. 2022Hierarchical3D Adapters for Long Video-to-text Summarizationcited by 12

50 most cited of 61 All 61 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 4; 2026: 7.

11 papers.

  1. 2025Summarizing Speech: A Comprehensive Surveycited by 5
  2. 2025Explanatory Summarization with Discourse-Driven Planningcited by 4
  3. 2026Cut to the Chase: Training-free Multimodal Summarization via Chain-of-Eventscited by 3
  4. 2026Disco-RAG: Discourse-Aware Retrieval-Augmented Generationcited by 1
  5. 2025Summarization of Multimodal Presentations with Vision-Language Models: Study of the Effect of Modalities and Structurecited by 0
  6. 2025Minimal Clips, Maximum Salience: Long Video Summarization via Key Moment Extractioncited by 0
  7. 2026LVSum: A Benchmark for Timestamp-Aware Long Video Summarizationcited by 0
  8. 2026Summarization is Not Dead Yetcited by 0
  9. 2026Helping Figures Tell their Story! Paper-Grounded Video Generation Explaining Complex Scientific Figurescited by 0
  10. 2026A Good Talk Does not Look Like a Summary, It Teaches You! Measuring Takeaways from Paper-to-Video Talkscited by 0
  11. 2026KnowVis: Knowledge-Centric Visual Summarization for Video Lecturescited by 0