# Two Stream Self-Supervised Learning for Action Recognition

arXiv: https://arxiv.org/abs/1806.07383 · HTML: https://mlsanity.com/paper/1806.07383/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Jun 2018.

## Built on by (most cited first, 2 of 2)

- [Distribution Estimation to Automate Transformation Policies for Self-Supervision](https://mlsanity.com/paper/2111.12265/) (2021)
- [JOSENet: A Joint Stream Embedding Network for Violence Detection in Surveillance Videos](https://mlsanity.com/paper/2405.02961/) (2024)

## Abstract (arXiv)

We present a self-supervised approach using spatio-temporal signals between video frames for action recognition. A two-stream architecture is leveraged to tangle spatial and temporal representation learning. Our task is formulated as both a sequence verification and spatio-temporal alignment tasks. The former task requires motion temporal structure understanding while the latter couples the learned motion with the spatial representation. The self-supervised pre-trained weights effectiveness is validated on the action recognition task. Quantitative evaluation shows the self-supervised approach competence on three datasets: HMDB51, UCF101, and Honda driving dataset (HDD). Further investigations to boost performance and generalize validity are still required.