# STAR-GNN: Spatial-Temporal Video Representation for Content-based Retrieval

arXiv: https://arxiv.org/abs/2208.06966 · HTML: https://mlsanity.com/paper/2208.06966/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 24 references that are in the map.
- First posted on arXiv 15 Aug 2022.

## Builds on (most cited first, 16 of 16)

- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [A Comprehensive Survey on Graph Neural Networks](https://mlsanity.com/paper/1901.00596/) (2019)
- [The Kinetics Human Action Video Dataset](https://mlsanity.com/paper/1705.06950/) (2017)
- [Simplifying Graph Convolutional Networks](https://mlsanity.com/paper/1902.07153/) (2019)
- [Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition](https://mlsanity.com/paper/1801.07455/) (2018)
- [Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networks](https://mlsanity.com/paper/1905.07953/) (2019)
- [Videos as Space-Time Region Graphs](https://mlsanity.com/paper/1806.01810/) (2018)
- [Particular object retrieval with integral max-pooling of CNN activations](https://mlsanity.com/paper/1511.05879/) (2015)
- [Aggregating Deep Convolutional Features for Image Retrieval](https://mlsanity.com/paper/1510.07493/) (2015)
- [Learning Multi-Granular Hypergraphs for Video-Based Person Re-Identification](https://mlsanity.com/paper/2104.14913/) (2021)
- [Resource Efficient 3D Convolutional Neural Networks](https://mlsanity.com/paper/1904.02422/) (2019)
- [ViSiL: Fine-grained Spatio-Temporal Video Similarity Learning](https://mlsanity.com/paper/1908.07410/) (2019)
- [Recurrent Space-time Graph Neural Networks](https://mlsanity.com/paper/1904.05582/) (2019)
- [Video Similarity and Alignment Learning on Partial Video Copy Detection](https://mlsanity.com/paper/2108.01817/) (2021)
- [Deep Learning Based Semantic Video Indexing and Retrieval](https://mlsanity.com/paper/1601.07754/) (2016)

## Abstract (arXiv)

We propose a video feature representation learning framework called STAR-GNN, which applies a pluggable graph neural network component on a multi-scale lattice feature graph. The essence of STAR-GNN is to exploit both the temporal dynamics and spatial contents as well as visual connections between regions at different scales in the frames. It models a video with a lattice feature graph in which the nodes represent regions of different granularity, with weighted edges that represent the spatial and temporal links. The contextual nodes are aggregated simultaneously by graph neural networks with parameters trained with retrieval triplet loss. In the experiments, we show that STAR-GNN effectively implements a dynamic attention mechanism on video frame sequences, resulting in the emphasis for dynamic and semantically rich content in the video, and is robust to noise and redundancies. Empirical results show that STAR-GNN achieves state-of-the-art performance for Content-Based Video Retrieval.