# VideoSAGE: Video Summarization with Graph Representation Learning

arXiv: https://arxiv.org/abs/2404.10539 · HTML: https://mlsanity.com/paper/2404.10539/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 34 references that are in the map.
- First posted on arXiv 14 Apr 2024.

## Builds on (most cited first, 23 of 23)

- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [Dynamic Graph CNN for Learning on Point Clouds](https://mlsanity.com/paper/1801.07829/) (2018)
- [Tune: A Research Platform for Distributed Model Selection and Training](https://mlsanity.com/paper/1807.05118/) (2018)
- [EGO-TOPO: Environment Affordances from Egocentric Video](https://mlsanity.com/paper/2001.04583/) (2020)
- [Deep Reinforcement Learning for Unsupervised Video Summarization with Diversity-Representativeness Reward](https://mlsanity.com/paper/1801.00054/) (2017)
- [CLIP-It! Language-Guided Video Summarization](https://mlsanity.com/paper/2107.00650/) (2021)
- [Summarizing Videos with Attention](https://mlsanity.com/paper/1812.01969/) (2018)
- [Align and Attend: Multimodal Summarization with Dual Contrastive Losses](https://mlsanity.com/paper/2303.07284/) (2023)
- [Video Summarization with Attention-Based Encoder-Decoder Networks](https://mlsanity.com/paper/1708.09545/) (2017)
- [Rethinking the Evaluation of Video Summaries](https://mlsanity.com/paper/1903.11328/) (2019)
- [A Structured Model For Action Detection](https://mlsanity.com/paper/1812.03544/) (2018)
- [LoGAN: Latent Graph Co-Attention Network for Weakly-Supervised Video Moment Retrieval](https://mlsanity.com/paper/1909.13784/) (2019)
- [Discriminative Feature Learning for Unsupervised Video Summarization](https://mlsanity.com/paper/1811.09791/) (2018)
- [Reconstructive Sequence-Graph Network for Video Summarization](https://mlsanity.com/paper/2105.04066/) (2021)
- [SumGraph: Video Summarization via Recursive Graph Modeling](https://mlsanity.com/paper/2007.08809/) (2020)
- [Space-Time Crop & Attend: Improving Cross-modal Video Representation Learning](https://mlsanity.com/paper/2103.10211/) (2021)
- [Learning Long-Term Spatial-Temporal Graphs for Active Speaker Detection](https://mlsanity.com/paper/2207.07783/) (2022)
- [Supervised Video Summarization via Multiple Feature Sets with Parallel Attention](https://mlsanity.com/paper/2104.11530/) (2021)
- [Intel Labs at Ego4D Challenge 2022: A Better Baseline for Audio-Visual Diarization](https://mlsanity.com/paper/2210.07764/) (2022)
- [Dynamic Graph Representation Learning for Video Dialog via Multi-Modal Shuffled Transformers](https://mlsanity.com/paper/2007.03848/) (2020)
- [STHG: Spatial-Temporal Heterogeneous Graph Learning for Advanced Audio-Visual Diarization](https://mlsanity.com/paper/2306.10608/) (2023)
- [Dynamic Emotion Modeling with Learnable Graphs and Graph Inception Network](https://mlsanity.com/paper/2008.02661/) (2020)

## Built on by (most cited first, 4 of 4)

- [Spiking Variational Graph Representation Inference for Video Summarization](https://mlsanity.com/paper/2508.15389/) (2025)
- [HierSum: A Global and Local Attention Mechanism for Video Summarization](https://mlsanity.com/paper/2504.18689/) (2025)
- [Language-Guided Graph Representation Learning for Video Summarization](https://mlsanity.com/paper/2511.10953/) (2025)
- [TPA: Temporal Prompt Alignment for Fetal Congenital Heart Defect Classification](https://mlsanity.com/paper/2508.15298/) (2025)

## Abstract (arXiv)

We propose a graph-based representation learning framework for video summarization. First, we convert an input video to a graph where nodes correspond to each of the video frames. Then, we impose sparsity on the graph by connecting only those pairs of nodes that are within a specified temporal distance. We then formulate the video summarization task as a binary node classification problem, precisely classifying video frames whether they should belong to the output summary video. A graph constructed this way aims to capture long-range interactions among video frames, and the sparsity ensures the model trains without hitting the memory and compute bottleneck. Experiments on two datasets(SumMe and TVSum) demonstrate the effectiveness of the proposed nimble model compared to existing state-of-the-art summarization approaches while being one order of magnitude more efficient in compute time and memory