# Human-like Relational Models for Activity Recognition in Video

arXiv: https://arxiv.org/abs/2107.05319 · HTML: https://mlsanity.com/paper/2107.05319/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 14 references that are in the map.
- First posted on arXiv 12 Jul 2021.

## Builds on (most cited first, 11 of 11)

- [A Closer Look at Spatiotemporal Convolutions for Action Recognition](https://mlsanity.com/paper/1711.11248/) (2017)
- [The "something something" video database for learning and evaluating visual common sense](https://mlsanity.com/paper/1706.04261/) (2017)
- [TSM: Temporal Shift Module for Efficient Video Understanding](https://mlsanity.com/paper/1811.08383/) (2018)
- [Temporal Relational Reasoning in Videos](https://mlsanity.com/paper/1711.08496/) (2017)
- [Long-Term Feature Banks for Detailed Video Understanding](https://mlsanity.com/paper/1812.05038/) (2018)
- [Videos as Space-Time Region Graphs](https://mlsanity.com/paper/1806.01810/) (2018)
- [STM: SpatioTemporal and Motion Encoding for Action Recognition](https://mlsanity.com/paper/1908.02486/) (2019)
- [Something-Else: Compositional Action Recognition with Spatial-Temporal Interaction Networks](https://mlsanity.com/paper/1912.09930/) (2019)
- [D3D: Distilled 3D Networks for Video Action Recognition](https://mlsanity.com/paper/1812.08249/) (2018)
- [Temporal Reasoning Graph for Activity Recognition](https://mlsanity.com/paper/1908.09995/) (2019)
- [What can human minimal videos tell us about dynamic recognition models?](https://mlsanity.com/paper/2104.09447/) (2021)

## Built on by (most cited first, 1 of 1)

- [Interpretable Action Recognition on Hard to Classify Actions](https://mlsanity.com/paper/2409.13091/) (2024)

## Abstract (arXiv)

Video activity recognition by deep neural networks is impressive for many classes. However, it falls short of human performance, especially for challenging to discriminate activities. Humans differentiate these complex activities by recognising critical spatio-temporal relations among explicitly recognised objects and parts, for example, an object entering the aperture of a container. Deep neural networks can struggle to learn such critical relationships effectively. Therefore we propose a more human-like approach to activity recognition, which interprets a video in sequential temporal phases and extracts specific relationships among objects and hands in those phases. Random forest classifiers are learnt from these extracted relationships. We apply the method to a challenging subset of the something-something dataset and achieve a more robust performance against neural network baselines on challenging activities.