# Long Short-Term Relation Networks for Video Action Detection

arXiv: https://arxiv.org/abs/2003.14065 · HTML: https://mlsanity.com/paper/2003.14065/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Mar 2020.

## Built on by (most cited first, 10 of 10)

- [Motion-Focused Contrastive Learning of Video Representations](https://mlsanity.com/paper/2201.04029/) (2022)
- [Stand-Alone Inter-Frame Attention in Video Models](https://mlsanity.com/paper/2206.06931/) (2022)
- [ACGNet: Action Complement Graph Network for Weakly-supervised Temporal Action Localization](https://mlsanity.com/paper/2112.10977/) (2021)
- [Representing Videos as Discriminative Sub-graphs for Action Recognition](https://mlsanity.com/paper/2201.04027/) (2022)
- [Optimization Planning for 3D ConvNets](https://mlsanity.com/paper/2201.04021/) (2022)
- [Boosting Video Representation Learning with Multi-Faceted Integration](https://mlsanity.com/paper/2201.04023/) (2022)
- [A Survey on Deep Learning-based Spatio-temporal Action Detection](https://mlsanity.com/paper/2308.01618/) (2023)
- [Identity-aware Graph Memory Network for Action Detection](https://mlsanity.com/paper/2108.11559/) (2021)
- [MOFO: MOtion FOcused Self-Supervision for Video Understanding](https://mlsanity.com/paper/2308.12447/) (2023)
- [Free-Form Composition Networks for Egocentric Action Recognition](https://mlsanity.com/paper/2307.06527/) (2023)

## Abstract (arXiv)

It has been well recognized that modeling human-object or object-object relations would be helpful for detection task. Nevertheless, the problem is not trivial especially when exploring the interactions between human actor, object and scene (collectively as human-context) to boost video action detectors. The difficulty originates from the aspect that reliable relations in a video should depend on not only short-term human-context relation in the present clip but also the temporal dynamics distilled over a long-range span of the video. This motivates us to capture both short-term and long-term relations in a video. In this paper, we present a new Long Short-Term Relation Networks, dubbed as LSTR, that novelly aggregates and propagates relation to augment features for video action detection. Technically, Region Proposal Networks (RPN) is remoulded to first produce 3D bounding boxes, i.e., tubelets, in each video clip. LSTR then models short-term human-context interactions within each clip through spatio-temporal attention mechanism and reasons long-term temporal dynamics across video clips via Graph Convolutional Networks (GCN) in a cascaded manner. Extensive experiments are conducted on four benchmark datasets, and superior results are reported when comparing to state-of-the-art methods.