# Guess Where? Actor-Supervision for Spatiotemporal Action Localization

arXiv: https://arxiv.org/abs/1804.01824 · HTML: https://mlsanity.com/paper/1804.01824/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Apr 2018.

## Built on by (most cited first, 8 of 8)

- [Weakly-Supervised Spatio-Temporal Anomaly Detection in Surveillance Video](https://mlsanity.com/paper/2108.03825/) (2021)
- [Unified Keypoint-based Action Recognition Framework via Structured Keypoint Pooling](https://mlsanity.com/paper/2303.15270/) (2023)
- [End-to-End Semi-Supervised Learning for Video Action Detection](https://mlsanity.com/paper/2203.04251/) (2022)
- [Semi-supervised Active Learning for Video Action Detection](https://mlsanity.com/paper/2312.07169/) (2023)
- [Actor-centered Representations for Action Localization in Streaming Videos](https://mlsanity.com/paper/2104.14131/) (2021)
- [OmViD: Omni-supervised active learning for video action detection](https://mlsanity.com/paper/2508.13983/) (2025)
- [Dual Guidance Semi-Supervised Action Detection](https://mlsanity.com/paper/2507.21247/) (2025)
- [Skeleton-based Zero-Shot Spatio-Temporal Action Localization via Weakly-Supervised Pretraining](https://mlsanity.com/paper/2608.25701/) (2026)

## Abstract (arXiv)

This paper addresses the problem of spatiotemporal localization of actions in videos. Compared to leading approaches, which all learn to localize based on carefully annotated boxes on training video frames, we adhere to a weakly-supervised solution that only requires a video class label. We introduce an actor-supervised architecture that exploits the inherent compositionality of actions in terms of actor transformations, to localize actions. We make two contributions. First, we propose actor proposals derived from a detector for human and non-human actors intended for images, which is linked over time by Siamese similarity matching to account for actor deformations. Second, we propose an actor-based attention mechanism that enables the localization of the actions from action class labels and actor proposals and is end-to-end trainable. Experiments on three human and non-human action datasets show actor supervision is state-of-the-art for weakly-supervised action localization and is even competitive to some fully-supervised alternatives.