# IntentVizor: Towards Generic Query Guided Interactive Video Summarization

arXiv: https://arxiv.org/abs/2109.14834 · HTML: https://mlsanity.com/paper/2109.14834/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 56 references that are in the map.
- First posted on arXiv 30 Sep 2021.

## Builds on (most cited first, 30 of 32)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Dynamic Graph CNN for Learning on Point Clouds](https://mlsanity.com/paper/1801.07829/) (2018)
- [SlowFast Networks for Video Recognition](https://mlsanity.com/paper/1812.03982/) (2018)
- [Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour](https://mlsanity.com/paper/1706.02677/) (2017)
- [Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition](https://mlsanity.com/paper/1801.07455/) (2018)
- [VL-BERT: Pre-training of Generic Visual-Linguistic Representations](https://mlsanity.com/paper/1908.08530/) (2019)
- [VideoBERT: A Joint Model for Video and Language Representation Learning](https://mlsanity.com/paper/1904.01766/) (2019)
- [Unified Vision-Language Pre-Training for Image Captioning and VQA](https://mlsanity.com/paper/1909.11059/) (2019)
- [G-TAD: Sub-Graph Localization for Temporal Action Detection](https://mlsanity.com/paper/1911.11462/) (2019)
- [Videos as Space-Time Region Graphs](https://mlsanity.com/paper/1806.01810/) (2018)
- [Graph Convolutional Networks for Temporal Action Localization](https://mlsanity.com/paper/1909.03252/) (2019)
- [Video Summarization with Long Short-term Memory](https://mlsanity.com/paper/1605.08110/) (2016)
- [A Deep Relevance Matching Model for Ad-hoc Retrieval](https://mlsanity.com/paper/1711.08611/) (2017)
- [Spatio-Temporal Graph for Video Captioning with Knowledge Distillation](https://mlsanity.com/paper/2003.13942/) (2020)
- [Deep Reinforcement Learning for Unsupervised Video Summarization with Diversity-Representativeness Reward](https://mlsanity.com/paper/1801.00054/) (2017)
- [Summarizing Videos with Attention](https://mlsanity.com/paper/1812.01969/) (2018)
- [Query-Focused Video Summarization: Dataset, Evaluation, and A Memory Network Based Approach](https://mlsanity.com/paper/1707.04960/) (2017)
- [Understanding Human Gaze Communication by Spatio-Temporal Graph Reasoning](https://mlsanity.com/paper/1909.02144/) (2019)
- [Discriminative Feature Learning for Unsupervised Video Summarization](https://mlsanity.com/paper/1811.09791/) (2018)
- [SumGraph: Video Summarization via Recursive Graph Modeling](https://mlsanity.com/paper/2007.08809/) (2020)
- [Spatial-Temporal Correlation and Topology Learning for Person Re-Identification in Videos](https://mlsanity.com/paper/2104.08241/) (2021)
- [Stacked Spatio-Temporal Graph Convolutional Networks for Action Segmentation](https://mlsanity.com/paper/1811.10575/) (2018)
- [Movie Summarization via Sparse Graph Construction](https://mlsanity.com/paper/2012.07536/) (2020)
- [Convolutional Hierarchical Attention Network for Query-Focused Video Summarization](https://mlsanity.com/paper/2002.03740/) (2020)
- [Generic Intent Representation in Web Search](https://mlsanity.com/paper/1907.10710/) (2019)
- [Query-Conditioned Three-Player Adversarial Network for Video Summarization](https://mlsanity.com/paper/1807.06677/) (2018)
- [Diversity-aware Multi-Video Summarization](https://mlsanity.com/paper/1706.03123/) (2017)
- [DeepQAMVS: Query-Aware Hierarchical Pointer Networks for Multi-Video Summarization](https://mlsanity.com/paper/2105.06441/) (2021)
- [Unsupervised Video Summarization via Multi-source Features](https://mlsanity.com/paper/2105.12532/) (2021)

## Built on by (most cited first, 18 of 18)

- [UniVTG: Towards Unified Video-Language Temporal Grounding](https://mlsanity.com/paper/2307.16715/) (2023)
- [EgoVLPv2: Egocentric Video-Language Pre-training with Fusion in the Backbone](https://mlsanity.com/paper/2307.05463/) (2023)
- [An Outlook into the Future of Egocentric Vision](https://mlsanity.com/paper/2308.07123/) (2023)
- [Satori: Towards Proactive AR Assistant with Belief-Desire-Intention User Modeling](https://mlsanity.com/paper/2410.16668/) (2024)
- [Challenges and Trends in Egocentric Vision: A Survey](https://mlsanity.com/paper/2503.15275/) (2025)
- [Beyond Uncertainty: Evidential Deep Learning for Robust Video Temporal Grounding](https://mlsanity.com/paper/2408.16272/) (2024)
- [Every Shot Counts: Using Exemplars for Repetition Counting in Videos](https://mlsanity.com/paper/2403.18074/) (2024)
- [MMSummary: Multimodal Summary Generation for Fetal Ultrasound Video](https://mlsanity.com/paper/2408.03761/) (2024)
- [Charting the Future of AI in Project-Based Learning: A Co-Design Exploration with Students](https://mlsanity.com/paper/2401.14915/) (2024)
- [OmniSTVG: Toward Spatio-Temporal Omni-Object Video Grounding](https://mlsanity.com/paper/2503.10500/) (2025)
- [Conditional Modeling Based Automatic Video Summarization](https://mlsanity.com/paper/2311.12159/) (2023)
- [Multimodal Frame-Scoring Transformer for Video Summarization](https://mlsanity.com/paper/2207.01814/) (2022)
- [Personalized Video Summarization by Multimodal Video Understanding](https://mlsanity.com/paper/2411.03531/) (2024)
- [HierSum: A Global and Local Attention Mechanism for Video Summarization](https://mlsanity.com/paper/2504.18689/) (2025)
- [Prompts to Summaries: Zero-Shot Language-Guided Video Summarization with Large Language and Video Models](https://mlsanity.com/paper/2506.10807/) (2025)
- [Video Summarization Techniques: A Comprehensive Review](https://mlsanity.com/paper/2410.04449/) (2024)
- [Language-guided Recursive Spatiotemporal Graph Modeling for Video Summarization](https://mlsanity.com/paper/2509.05604/) (2025)
- [Identity-Centric Video Summarization via Hierarchical Fusion of Biometric, Appearance, and 3D Body Features](https://mlsanity.com/paper/2609.25837/) (2026)

## Abstract (arXiv)

The target of automatic video summarization is to create a short skim of the original long video while preserving the major content/events. There is a growing interest in the integration of user queries into video summarization or query-driven video summarization. This video summarization method predicts a concise synopsis of the original video based on the user query, which is commonly represented by the input text. However, two inherent problems exist in this query-driven way. First, the text query might not be enough to describe the exact and diverse needs of the user. Second, the user cannot edit once the summaries are produced, while we assume the needs of the user should be subtle and need to be adjusted interactively. To solve these two problems, we propose IntentVizor, an interactive video summarization framework guided by generic multi-modality queries. The input query that describes the user's needs are not limited to text but also the video snippets. We further represent these multi-modality finer-grained queries as user `intent', which is interpretable, interactable, editable, and can better quantify the user's needs. In this paper, we use a set of the proposed intents to represent the user query and design a new interactive visual analytic interface. Users can interactively control and adjust these mixed-initiative intents to obtain a more satisfying summary through the interface. Also, to improve the summarization quality via video understanding, a novel Granularity-Scalable Ego-Graph Convolutional Networks (GSE-GCN) is proposed. We conduct our experiments on two benchmark datasets. Comparisons with the state-of-the-art methods verify the effectiveness of the proposed framework. Code and dataset are available at https://github.com/jnzs1836/intent-vizor.