mlsanityData to 14 Oct 2026

Language-guided Recursive Spatiotemporal Graph Modeling for Video Summarization

arXiv 2509.05604 ↗First posted 6 Sep 2025cs.CV cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Video summarization aims to select keyframes that are visually diverse and can represent the whole story of a given video. Previous approaches have focused on global interlinkability between frames in a video by temporal modeling. However, fine-grained visual entities, such as objects, are also highly related to the main content of the video. Moreover, language-guided video summarization, which has recently been studied, requires a comprehensive linguistic understanding of complex real-world videos. To consider how all the objects are semantically related to each other, this paper regards video summarization as a language-guided spatiotemporal graph modeling problem. We present recursive spatiotemporal graph networks, called VideoGraph, which formulate the objects and frames as nodes of the spatial and temporal graphs, respectively. The nodes in each graph are connected and aggregated with graph edges, representing the semantic relationships between the nodes. To prevent the edges from being configured with visual similarity, we incorporate language queries derived from the video into the graph node representations, enabling them to contain semantic knowledge. In addition, we adopt a recursive strategy to refine initial graphs and correctly classify each frame node as a keyframe. In our experiments, VideoGraph achieves state-of-the-art performance on several benchmarks for generic and query-focused video summarization in both supervised and unsupervised manners. The code is available at https://github.com/park-jungin/videograph.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 58 of its 102 references that are in the map. First posted on arXiv 6 Sep 2025.

Builds on

58 of 102 references.

References · 58

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  4. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  5. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
  6. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  7. 2014Going Deeper with Convolutionscited by 4,241
  8. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,241
  9. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,656
  10. 2017Non-local Neural Networkscited by 2,293
  11. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,555
  12. 2017Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answeringcited by 1,312
38 more
  1. 2018Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognitioncited by 1,209
  2. 2017Spatio-Temporal Graph Convolutional Networks: A Deep Learning Framework for Traffic Forecastingcited by 1,136
  3. 2021VideoCLIP: Contrastive Pre-training for Zero-shot Video-Text Understandingcited by 602
  4. 2020Point-GNN: Graph Neural Network for 3D Object Detection in a Point Cloudcited by 335
  5. 2015Structural-RNN: Deep Learning on Spatio-Temporal Graphscited by 234
  6. 2018Videos as Space-Time Region Graphscited by 221
  7. 2019Graph Convolutional Networks for Temporal Action Localizationcited by 208
  8. 2016Video Summarization with Long Short-term Memorycited by 142
  9. 2015Human Pose Estimation with Iterative Error Feedbackcited by 121
  10. 2020Where Does It Exist: Spatio-Temporal Video Grounding for Multi-Form Sentencescited by 118
  11. 2017Learning non-maximum suppressioncited by 113
  12. 2020Spatio-Temporal Graph for Video Captioning with Knowledge Distillationcited by 109
  13. 2020Object Relational Graph with Teacher-Recommended Learning for Video Captioningcited by 107
  14. 2017Deep Reinforcement Learning for Unsupervised Video Summarization with Diversity-Representativeness Rewardcited by 105
  15. 2021CLIP-It! Language-Guided Video Summarizationcited by 99
  16. 2020Graph Structured Network for Image-Text Matchingcited by 93
  17. 2020Dynamic GCN: Context-enriched Topology Learning for Skeleton-based Action Recognitioncited by 82
  18. 2020A Better Use of Audio-Visual Cues: Dense Video Captioning with Bi-modal Transformercited by 76
  19. 2018Video Summarization Using Fully Convolutional Sequence Networkscited by 71
  20. 2021Enriching Local and Global Contexts for Temporal Action Localizationcited by 64
  21. 2017Video Summarization with Attention-Based Encoder-Decoder Networkscited by 63
  22. 2019Rethinking the Evaluation of Video Summariescited by 62
  23. 2021Bridge to Answer: Structure-aware Graph Interaction Network for Video Question Answeringcited by 56
  24. 2021Target Adaptive Context Aggregation for Video Scene Graph Generationcited by 52
  25. 2019Hierarchical Recurrent Neural Network for Video Summarizationcited by 44
  26. 2017Query-Focused Video Summarization: Dataset, Evaluation, and A Memory Network Based Approachcited by 42
  27. 2018Video Summarization by Learning from Unpaired Datacited by 36
  28. 2019STEP: Spatial Temporal Graph Convolutional Networks for Emotion Perception from Gaitscited by 36
  29. 2021Reconstructive Sequence-Graph Network for Video Summarizationcited by 34
  30. 2020SumGraph: Video Summarization via Recursive Graph Modelingcited by 32
  31. 2021Graph Convolutional Module for Temporal Action Localization in Videoscited by 29
  32. 2016Query-Focused Extractive Video Summarizationcited by 26
  33. 2016Summary Transfer: Exemplar-based Subset Selection for Video Summarizationcited by 24
  34. 2019Cycle-SUM: Cycle-consistent Adversarial LSTM Networks for Unsupervised Video Summarizationcited by 24
  35. 2017Temporal Dynamic Graph LSTM for Action-driven Video Object Detectioncited by 23
  36. 2021Classification-Then-Grounding: Reformulating Video Scene Graphs as Temporal Bipartite Graphscited by 23
  37. 2022Progressive Video Summarization via Multimodal Self-supervised Learningcited by 21
  38. 2021IntentVizor: Towards Generic Query Guided Interactive Video Summarizationcited by 18

50 most cited of 58 All 58 (JSON)

Built on by

Not cited yet.