# Weakly Supervised Temporal Action Localization with Segment-Level Labels

arXiv: https://arxiv.org/abs/2007.01598 · HTML: https://mlsanity.com/paper/2007.01598/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Jul 2020.

## Built on by (most cited first, 9 of 9)

- [Deep Learning-based Action Detection in Untrimmed Videos: A Survey](https://mlsanity.com/paper/2110.00111/) (2021)
- [Support-Set Based Cross-Supervision for Video Grounding](https://mlsanity.com/paper/2108.10576/) (2021)
- [Exploring Segment-level Semantics for Online Phase Recognition from Surgical Videos](https://mlsanity.com/paper/2111.11044/) (2021)
- [Less is More: Surgical Phase Recognition from Timestamp Supervision](https://mlsanity.com/paper/2202.08199/) (2022)
- [DDG-Net: Discriminability-Driven Graph Network for Weakly-supervised Temporal Action Localization](https://mlsanity.com/paper/2307.16415/) (2023)
- [SVGraph: Learning Semantic Graphs from Instructional Videos](https://mlsanity.com/paper/2207.08001/) (2022)
- [Weakly-Supervised Temporal Action Localization with Bidirectional Semantic Consistency Constraint](https://mlsanity.com/paper/2304.12616/) (2023)
- [Active Learning with Effective Scoring Functions for Semi-Supervised Temporal Action Localization](https://mlsanity.com/paper/2208.14856/) (2022)
- [Dilation-Erosion for Single-Frame Supervised Temporal Action Localization](https://mlsanity.com/paper/2212.06348/) (2022)

## Abstract (arXiv)

Temporal action localization presents a trade-off between test performance and annotation-time cost. Fully supervised methods achieve good performance with time-consuming boundary annotations. Weakly supervised methods with cheaper video-level category label annotations result in worse performance. In this paper, we introduce a new segment-level supervision setting: segments are labeled when annotators observe actions happening here. We incorporate this segment-level supervision along with a novel localization module in the training. Specifically, we devise a partial segment loss regarded as a loss sampling to learn integral action parts from labeled segments. Since the labeled segments are only parts of actions, the model tends to overfit along with the training process. To tackle this problem, we first obtain a similarity matrix from discriminative features guided by a sphere loss. Then, a propagation loss is devised based on the matrix to act as a regularization term, allowing implicit unlabeled segments propagation during training. Experiments validate that our method can outperform the video-level supervision methods with almost same the annotation time.