# Spot On: Action Localization from Pointly-Supervised Proposals

arXiv: https://arxiv.org/abs/1604.07602 · HTML: https://mlsanity.com/paper/1604.07602/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Apr 2016.

## Built on by (most cited first, 17 of 17)

- [Pointly-Supervised Instance Segmentation](https://mlsanity.com/paper/2104.06404/) (2021)
- [Holmes-VAD: Towards Unbiased and Explainable Video Anomaly Detection via Multi-modal LLM](https://mlsanity.com/paper/2406.12235/) (2024)
- [Temporal Action Segmentation from Timestamp Supervision](https://mlsanity.com/paper/2103.06669/) (2021)
- [Learning Action Completeness from Points for Weakly-supervised Temporal Action Localization](https://mlsanity.com/paper/2108.05029/) (2021)
- [How Severe is Benchmark-Sensitivity in Video Self-Supervised Learning?](https://mlsanity.com/paper/2203.14221/) (2022)
- [Background-Click Supervision for Temporal Action Localization](https://mlsanity.com/paper/2111.12449/) (2021)
- [Constraint and Union for Partially-Supervised Temporal Sentence Grounding](https://mlsanity.com/paper/2302.09850/) (2023)
- [Less is More: Surgical Phase Recognition from Timestamp Supervision](https://mlsanity.com/paper/2202.08199/) (2022)
- [Panoramic Human Activity Recognition](https://mlsanity.com/paper/2203.03806/) (2022)
- [Objects do not disappear: Video object detection by single-frame object location anticipation](https://mlsanity.com/paper/2308.04770/) (2023)
- [GlanceVAD: Exploring Glance Supervision for Label-efficient Video Anomaly Detection](https://mlsanity.com/paper/2403.06154/) (2024)
- [Point-VOS: Pointing Up Video Object Segmentation](https://mlsanity.com/paper/2402.05917/) (2024)
- [SEVERE++: Evaluating Benchmark Sensitivity in Generalization of Video Representation Learning](https://mlsanity.com/paper/2504.05706/) (2025)
- [Faster Video Moment Retrieval with Point-Level Supervision](https://mlsanity.com/paper/2305.14017/) (2023)
- [Learning Temporal Sentence Grounding From Narrated EgoVideos](https://mlsanity.com/paper/2310.17395/) (2023)
- [Autonomous Curiosity for Real-Time Training Onboard Robotic Agents](https://mlsanity.com/paper/2109.00927/) (2021)
- [Actor-identified Spatiotemporal Action Detection -- Detecting Who Is Doing What in Videos](https://mlsanity.com/paper/2208.12940/) (2022)

## Abstract (arXiv)

We strive for spatio-temporal localization of actions in videos. The state-of-the-art relies on action proposals at test time and selects the best one with a classifier trained on carefully annotated box annotations. Annotating action boxes in video is cumbersome, tedious, and error prone. Rather than annotating boxes, we propose to annotate actions in video with points on a sparse subset of frames only. We introduce an overlap measure between action proposals and points and incorporate them all into the objective of a non-convex Multiple Instance Learning optimization. Experimental evaluation on the UCF Sports and UCF 101 datasets shows that (i) spatio-temporal proposals can be used to train classifiers while retaining the localization performance, (ii) point annotations yield results comparable to box annotations while being significantly faster to annotate, (iii) with a minimum amount of supervision our approach is competitive to the state-of-the-art. Finally, we introduce spatio-temporal action annotations on the train and test videos of Hollywood2, resulting in Hollywood2Tubes, available at http://tinyurl.com/hollywood2tubes.