# Video action detection by learning graph-based spatio-temporal interactions

arXiv: https://arxiv.org/abs/1912.04316 · HTML: https://mlsanity.com/paper/1912.04316/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Dec 2019.

## Built on by (most cited first, 4 of 4)

- [Deep Learning-based Action Detection in Untrimmed Videos: A Survey](https://mlsanity.com/paper/2110.00111/) (2021)
- [E^2TAD: An Energy-Efficient Tracking-based Action Detector](https://mlsanity.com/paper/2204.04416/) (2022)
- [A Grammatical Compositional Model for Video Action Detection](https://mlsanity.com/paper/2310.02887/) (2023)
- [Dual Guidance Semi-Supervised Action Detection](https://mlsanity.com/paper/2507.21247/) (2025)

## Abstract (arXiv)

Action Detection is a complex task that aims to detect and classify human actions in video clips. Typically, it has been addressed by processing fine-grained features extracted from a video classification backbone. Recently, thanks to the robustness of object and people detectors, a deeper focus has been added on relationship modelling. Following this line, we propose a graph-based framework to learn high-level interactions between people and objects, in both space and time. In our formulation, spatio-temporal relationships are learned through self-attention on a multi-layer graph structure which can connect entities from consecutive clips, thus considering long-range spatial and temporal dependencies. The proposed module is backbone independent by design and does not require end-to-end training. Extensive experiments are conducted on the AVA dataset, where our model demonstrates state-of-the-art results and consistent improvements over baselines built with different backbones. Code is publicly available at https://github.com/aimagelab/STAGE_action_detection.