# What Is That Talk About? A Video-to-Text Summarization Dataset for Scientific Presentations

arXiv: https://arxiv.org/abs/2502.08279 · HTML: https://mlsanity.com/paper/2502.08279/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 61 of its 97 references that are in the map.
- First posted on arXiv 12 Feb 2025.

## Builds on (most cited first, 30 of 61)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [QLoRA: Efficient Finetuning of Quantized LLMs](https://mlsanity.com/paper/2305.14314/) (2023)
- [MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI](https://mlsanity.com/paper/2311.16502/) (2023)
- [CIDEr: Consensus-based Image Description Evaluation](https://mlsanity.com/paper/1411.5726/) (2014)
- [A Call for Clarity in Reporting BLEU Scores](https://mlsanity.com/paper/1804.08771/) (2018)
- [MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts](https://mlsanity.com/paper/2310.02255/) (2023)
- [Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understanding](https://mlsanity.com/paper/2306.02858/) (2023)
- [Video-LLaVA: Learning United Visual Representation by Alignment Before Projection](https://mlsanity.com/paper/2311.10122/) (2023)
- [When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories](https://mlsanity.com/paper/2212.10511/) (2022)
- [A Diversity-Promoting Objective Function for Neural Conversation Models](https://mlsanity.com/paper/1510.03055/) (2015)
- [BLINK: Multimodal Large Language Models Can See but Not Perceive](https://mlsanity.com/paper/2404.12390/) (2024)
- [Hallucination of Multimodal Large Language Models: A Survey](https://mlsanity.com/paper/2404.18930/) (2024)
- [MLLM-as-a-Judge: Assessing Multimodal LLM-as-a-Judge with Vision-Language Benchmark](https://mlsanity.com/paper/2402.04788/) (2024)
- [VideoScore: Building Automatic Metrics to Simulate Fine-grained Human Feedback for Video Generation](https://mlsanity.com/paper/2406.15252/) (2024)
- [Aligning with Human Judgement: The Role of Pairwise Preference in Large Language Model Evaluators](https://mlsanity.com/paper/2403.16950/) (2024)
- [How2: A Large-scale Dataset for Multimodal Language Understanding](https://mlsanity.com/paper/1811.00347/) (2018)
- [TLDR: Extreme Summarization of Scientific Documents](https://mlsanity.com/paper/2004.15011/) (2020)
- [Grounding Language Models to Images for Multimodal Inputs and Outputs](https://mlsanity.com/paper/2301.13823/) (2023)
- [A Comprehensive Survey of Scientific Large Language Models and Their Applications in Scientific Discovery](https://mlsanity.com/paper/2406.10833/) (2024)
- [Video ReCap: Recursive Captioning of Hour-Long Videos](https://mlsanity.com/paper/2402.13250/) (2024)
- [Planning with Learned Entity Prompts for Abstractive Summarization](https://mlsanity.com/paper/2104.07606/) (2021)
- [SPIQA: A Dataset for Multimodal Question Answering on Scientific Papers](https://mlsanity.com/paper/2407.09413/) (2024)
- [Align and Attend: Multimodal Summarization with Dual Contrastive Losses](https://mlsanity.com/paper/2303.07284/) (2023)
- [Shot2Story: A New Benchmark for Comprehensive Understanding of Multi-shot Videos](https://mlsanity.com/paper/2312.10300/) (2023)
- [From Pixels to Insights: A Survey on Automatic Chart Understanding in the Era of Large Foundation Models](https://mlsanity.com/paper/2403.12027/) (2024)
- [ChatGPT vs Human-authored Text: Insights into Controllable Text Summarization and Sentence Style Transfer](https://mlsanity.com/paper/2306.07799/) (2023)
- [V2Xum-LLM: Cross-Modal Video Summarization with Temporal Prompt Instruction Tuning](https://mlsanity.com/paper/2404.12353/) (2024)

## Built on by (most cited first, 11 of 11)

- [Summarizing Speech: A Comprehensive Survey](https://mlsanity.com/paper/2504.08024/) (2025)
- [Explanatory Summarization with Discourse-Driven Planning](https://mlsanity.com/paper/2504.19339/) (2025)
- [Cut to the Chase: Training-free Multimodal Summarization via Chain-of-Events](https://mlsanity.com/paper/2603.06213/) (2026)
- [Disco-RAG: Discourse-Aware Retrieval-Augmented Generation](https://mlsanity.com/paper/2601.04377/) (2026)
- [Summarization of Multimodal Presentations with Vision-Language Models: Study of the Effect of Modalities and Structure](https://mlsanity.com/paper/2504.10049/) (2025)
- [Minimal Clips, Maximum Salience: Long Video Summarization via Key Moment Extraction](https://mlsanity.com/paper/2512.11399/) (2025)
- [LVSum: A Benchmark for Timestamp-Aware Long Video Summarization](https://mlsanity.com/paper/2604.10024/) (2026)
- [Summarization is Not Dead Yet](https://mlsanity.com/paper/2606.08000/) (2026)
- [Helping Figures Tell their Story! Paper-Grounded Video Generation Explaining Complex Scientific Figures](https://mlsanity.com/paper/2606.12576/) (2026)
- [A Good Talk Does not Look Like a Summary, It Teaches You! Measuring Takeaways from Paper-to-Video Talks](https://mlsanity.com/paper/2606.28531/) (2026)
- [KnowVis: Knowledge-Centric Visual Summarization for Video Lectures](https://mlsanity.com/paper/2609.03742/) (2026)

## Abstract (arXiv)

Transforming recorded videos into concise and accurate textual summaries is a growing challenge in multimodal learning. This paper introduces VISTA, a dataset specifically designed for video-to-text summarization in scientific domains. VISTA contains 18,599 recorded AI conference presentations paired with their corresponding paper abstracts. We benchmark the performance of state-of-the-art large models and apply a plan-based framework to better capture the structured nature of abstracts. Both human and automated evaluations confirm that explicit planning enhances summary quality and factual consistency. However, a considerable gap remains between models and human performance, highlighting the challenges of our dataset. This study aims to pave the way for future research on scientific video-to-text summarization.