# Spatial-Temporal Human-Object Interaction Detection

arXiv: https://arxiv.org/abs/2508.17270 · HTML: https://mlsanity.com/paper/2508.17270/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 24 references that are in the map.
- First posted on arXiv 24 Aug 2025.

## Builds on (most cited first, 11 of 11)

- [VQA: Visual Question Answering](https://mlsanity.com/paper/1505.00468/) (2015)
- [AVA: A Video Dataset of Spatio-temporally Localized Atomic Visual Actions](https://mlsanity.com/paper/1705.08421/) (2017)
- [RMPE: Regional Multi-person Pose Estimation](https://mlsanity.com/paper/1612.00137/) (2016)
- [Temporal Relational Reasoning in Videos](https://mlsanity.com/paper/1711.08496/) (2017)
- [Learning to Detect Human-Object Interactions](https://mlsanity.com/paper/1702.05448/) (2017)
- [Learning Human-Object Interactions by Graph Parsing Neural Networks](https://mlsanity.com/paper/1808.07962/) (2018)
- [iCAN: Instance-Centric Attention Network for Human-Object Interaction Detection](https://mlsanity.com/paper/1808.10437/) (2018)
- [Actor-Centric Relation Network](https://mlsanity.com/paper/1807.10982/) (2018)
- [Object Level Visual Reasoning in Videos](https://mlsanity.com/paper/1806.06157/) (2018)
- [Online Real-time Multiple Spatiotemporal Action Localisation and Prediction](https://mlsanity.com/paper/1611.08563/) (2016)
- [Tube Convolutional Neural Network (T-CNN) for Action Detection in Videos](https://mlsanity.com/paper/1703.10664/) (2017)

## Abstract (arXiv)

In this paper, we propose a new instance-level human-object interaction detection task on videos called ST-HOID, which aims to distinguish fine-grained human-object interactions (HOIs) and the trajectories of subjects and objects. It is motivated by the fact that HOI is crucial for human-centric video content understanding. To solve ST-HOID, we propose a novel method consisting of an object trajectory detection module and an interaction reasoning module. Furthermore, we construct the first dataset named VidOR-HOID for ST-HOID evaluation, which contains 10,831 spatial-temporal HOI instances. We conduct extensive experiments to evaluate the effectiveness of our method. The experimental results demonstrate that our method outperforms the baselines generated by the state-of-the-art methods of image human-object interaction detection, video visual relation detection and video human-object interaction recognition.