# Human Action Localization with Sparse Spatial Supervision

arXiv: https://arxiv.org/abs/1605.05197 · HTML: https://mlsanity.com/paper/1605.05197/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 May 2016.

## Built on by (most cited first, 8 of 8)

- [MultiSports: A Multi-Person Video Dataset of Spatio-Temporally Localized Sports Actions](https://mlsanity.com/paper/2105.07404/) (2021)
- [FineAction: A Fine-Grained Video Dataset for Temporal Action Localization](https://mlsanity.com/paper/2105.11107/) (2021)
- [SportsHHI: A Dataset for Human-Human Interaction Detection in Sports Videos](https://mlsanity.com/paper/2404.04565/) (2024)
- [Panoramic Human Activity Recognition](https://mlsanity.com/paper/2203.03806/) (2022)
- [Simitate: A Hybrid Imitation Learning Benchmark](https://mlsanity.com/paper/1905.06002/) (2019)
- [Discovering Multi-Label Actor-Action Association in a Weakly Supervised Setting](https://mlsanity.com/paper/2101.08567/) (2021)
- [Hierarchical Graph-RNNs for Action Detection of Multiple Activities](https://mlsanity.com/paper/2101.08581/) (2021)
- [Spot What Matters: Learning Context Using Graph Convolutional Networks for Weakly-Supervised Action Detection](https://mlsanity.com/paper/2107.13648/) (2021)

## Abstract (arXiv)

We introduce an approach for spatio-temporal human action localization using sparse spatial supervision. Our method leverages the large amount of annotated humans available today and extracts human tubes by combining a state-of-the-art human detector with a tracking-by-detection approach. Given these high-quality human tubes and temporal supervision, we select positive and negative tubes with very sparse spatial supervision, i.e., only one spatially annotated frame per instance. The selected tubes allow us to effectively learn a spatio-temporal action detector based on dense trajectories or CNNs. We conduct experiments on existing action localization benchmarks: UCF-Sports, J-HMDB and UCF-101. Our results show that our approach, despite using sparse spatial supervision, performs on par with methods using full supervision, i.e., one bounding box annotation per frame. To further validate our method, we introduce DALY (Daily Action Localization in YouTube), a dataset for realistic action localization in space and time. It contains high quality temporal and spatial annotations for 3.6k instances of 10 actions in 31 hours of videos (3.3M frames). It is an order of magnitude larger than existing datasets, with more diversity in appearance and long untrimmed videos.