# Query-adaptive Video Summarization via Quality-aware Relevance Estimation

arXiv: https://arxiv.org/abs/1705.00581 · HTML: https://mlsanity.com/paper/1705.00581/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 May 2017.

## Built on by (most cited first, 22 of 22)

- [Video Summarization Using Deep Neural Networks: A Survey](https://mlsanity.com/paper/2101.06072/) (2021)
- [PRISM: A Rich Class of Parameterized Submodular Information Measures for Guided Subset Selection](https://mlsanity.com/paper/2103.00128/) (2021)
- [Submodularity In Machine Learning and Artificial Intelligence](https://mlsanity.com/paper/2202.00132/) (2022)
- [GPT2MVS: Generative Pre-trained Transformer-2 for Multi-modal Video Summarization](https://mlsanity.com/paper/2104.12465/) (2021)
- [Learning Pixel-Level Distinctions for Video Highlight Detection](https://mlsanity.com/paper/2204.04615/) (2022)
- [Query-based Video Summarization with Pseudo Label Supervision](https://mlsanity.com/paper/2307.01945/) (2023)
- [CLINICAL: Targeted Active Learning for Imbalanced Medical Image Classification](https://mlsanity.com/paper/2210.01520/) (2022)
- [Submodular Mutual Information for Targeted Data Subset Selection](https://mlsanity.com/paper/2105.00043/) (2021)
- [DeepQAMVS: Query-Aware Hierarchical Pointer Networks for Multi-Video Summarization](https://mlsanity.com/paper/2105.06441/) (2021)
- [Submodlib: A Submodular Optimization Library](https://mlsanity.com/paper/2202.10680/) (2022)
- [Causalainer: Causal Explainer for Automatic Video Summarization](https://mlsanity.com/paper/2305.00455/) (2023)
- [How Good is a Video Summary? A New Benchmarking Dataset and Evaluation Framework Towards Realistic Video Summarization](https://mlsanity.com/paper/2101.10514/) (2021)
- [Video Summarization Overview](https://mlsanity.com/paper/2210.11707/) (2022)
- [Conditional Modeling Based Automatic Video Summarization](https://mlsanity.com/paper/2311.12159/) (2023)
- [A Lightweight Moment Retrieval System with Global Re-Ranking and Robust Adaptive Bidirectional Temporal Search](https://mlsanity.com/paper/2504.09298/) (2025)
- [Causal Video Summarizer for Video Exploration](https://mlsanity.com/paper/2307.01947/) (2023)
- [Personalized Video Summarization by Multimodal Video Understanding](https://mlsanity.com/paper/2411.03531/) (2024)
- [SD-VSum: A Method and Dataset for Script-Driven Video Summarization](https://mlsanity.com/paper/2505.03319/) (2025)
- [Show Me What I Like: Detecting User-Specific Video Highlights Using Content-Based Multi-Head Attention](https://mlsanity.com/paper/2207.08352/) (2022)
- [BASIL: Balanced Active Semi-supervised Learning for Class Imbalanced Datasets](https://mlsanity.com/paper/2203.05651/) (2022)
- [SD-MVSum: Script-Driven Multimodal Video Summarization Method and Datasets](https://mlsanity.com/paper/2510.05652/) (2025)
- [AdSum: Two-stream Audio-visual Summarization for Automated Video Advertisement Clipping](https://mlsanity.com/paper/2510.26569/) (2025)

## Abstract (arXiv)

Although the problem of automatic video summarization has recently received a lot of attention, the problem of creating a video summary that also highlights elements relevant to a search query has been less studied. We address this problem by posing query-relevant summarization as a video frame subset selection problem, which lets us optimise for summaries which are simultaneously diverse, representative of the entire video, and relevant to a text query. We quantify relevance by measuring the distance between frames and queries in a common textual-visual semantic embedding space induced by a neural network. In addition, we extend the model to capture query-independent properties, such as frame quality. We compare our method against previous state of the art on textual-visual embeddings for thumbnail selection and show that our model outperforms them on relevance prediction. Furthermore, we introduce a new dataset, annotated with diversity and query-specific relevance labels. On this dataset, we train and test our complete model for video summarization and show that it outperforms standard baselines such as Maximal Marginal Relevance.