# SD-VSum: A Method and Dataset for Script-Driven Video Summarization

arXiv: https://arxiv.org/abs/2505.03319 · HTML: https://mlsanity.com/paper/2505.03319/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 36 references that are in the map.
- First posted on arXiv 6 May 2025.

## Builds on (most cited first, 18 of 18)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Dense-Captioning Events in Videos](https://mlsanity.com/paper/1705.00754/) (2017)
- [CLIP-It! Language-Guided Video Summarization](https://mlsanity.com/paper/2107.00650/) (2021)
- [Video Summarization Using Deep Neural Networks: A Survey](https://mlsanity.com/paper/2101.06072/) (2021)
- [Align and Attend: Multimodal Summarization with Dual Contrastive Losses](https://mlsanity.com/paper/2303.07284/) (2023)
- [Rethinking the Evaluation of Video Summaries](https://mlsanity.com/paper/1903.11328/) (2019)
- [Query-Focused Video Summarization: Dataset, Evaluation, and A Memory Network Based Approach](https://mlsanity.com/paper/1707.04960/) (2017)
- [VideoXum: Cross-modal Visual and Textural Summarization of Videos](https://mlsanity.com/paper/2303.12060/) (2023)
- [TL;DW? Summarizing Instructional Videos with Task Relevance & Cross-Modal Saliency](https://mlsanity.com/paper/2208.06773/) (2022)
- [A Survey on Temporal Sentence Grounding in Videos](https://mlsanity.com/paper/2109.08039/) (2021)
- [Query-Focused Extractive Video Summarization](https://mlsanity.com/paper/1607.05177/) (2016)
- [Query-controllable Video Summarization](https://mlsanity.com/paper/2004.03661/) (2020)
- [Query-adaptive Video Summarization via Quality-aware Relevance Estimation](https://mlsanity.com/paper/1705.00581/) (2017)
- [Scaling Up Video Summarization Pretraining with Large Language Models](https://mlsanity.com/paper/2404.03398/) (2024)
- [Convolutional Hierarchical Attention Network for Query-Focused Video Summarization](https://mlsanity.com/paper/2002.03740/) (2020)
- [Query-based Video Summarization with Pseudo Label Supervision](https://mlsanity.com/paper/2307.01945/) (2023)
- [Query-Conditioned Three-Player Adversarial Network for Video Summarization](https://mlsanity.com/paper/1807.06677/) (2018)
- [VideoSET: Video Summary Evaluation through Text](https://mlsanity.com/paper/1406.5824/) (2014)

## Built on by (most cited first, 3 of 3)

- [TSalV360: A Method and Dataset for Text-driven Saliency Detection in 360-Degrees Videos](https://mlsanity.com/paper/2509.26208/) (2025)
- [SD-MVSum: Script-Driven Multimodal Video Summarization Method and Datasets](https://mlsanity.com/paper/2510.05652/) (2025)
- [LVSum: A Benchmark for Timestamp-Aware Long Video Summarization](https://mlsanity.com/paper/2604.10024/) (2026)

## Abstract (arXiv)

In this work, we introduce the task of script-driven video summarization, which aims to produce a summary of the full-length video by selecting the parts that are most relevant to a user-provided script outlining the visual content of the desired summary. Following, we extend a recently-introduced large-scale dataset for generic video summarization (VideoXum) by producing natural language descriptions of the different human-annotated summaries that are available per video. In this way we make it compatible with the introduced task, since the available triplets of ``video, summary and summary description'' can be used for training a method that is able to produce different summaries for a given video, driven by the provided script about the content of each summary. Finally, we develop a new network architecture for script-driven video summarization (SD-VSum), that employs a cross-modal attention mechanism for aligning and fusing information from the visual and text modalities. Our experimental evaluations demonstrate the advanced performance of SD-VSum against SOTA approaches for query-driven and generic (unimodal and multimodal) summarization from the literature, and document its capacity to produce video summaries that are adapted to each user's needs about their content.