# Active Learning with Effective Scoring Functions for Semi-Supervised Temporal Action Localization

arXiv: https://arxiv.org/abs/2208.14856 · HTML: https://mlsanity.com/paper/2208.14856/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 29 of its 54 references that are in the map.
- First posted on arXiv 31 Aug 2022.

## Builds on (most cited first, 29 of 29)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [A Survey of Deep Active Learning](https://mlsanity.com/paper/2009.00236/) (2020)
- [Soft-NMS -- Improving Object Detection With One Line of Code](https://mlsanity.com/paper/1704.04503/) (2017)
- [Learning Loss for Active Learning](https://mlsanity.com/paper/1905.03677/) (2019)
- [BMN: Boundary-Matching Network for Temporal Action Proposal Generation](https://mlsanity.com/paper/1907.09702/) (2019)
- [BSN: Boundary Sensitive Network for Temporal Action Proposal Generation](https://mlsanity.com/paper/1806.02964/) (2018)
- [Graph Convolutional Networks for Temporal Action Localization](https://mlsanity.com/paper/1909.03252/) (2019)
- [Rethinking the Faster R-CNN Architecture for Temporal Action Localization](https://mlsanity.com/paper/1804.07667/) (2018)
- [Cost-Effective Active Learning for Deep Image Classification](https://mlsanity.com/paper/1701.03551/) (2017)
- [Human Action Recognition from Various Data Modalities: A Review](https://mlsanity.com/paper/2012.11866/) (2020)
- [R-C3D: Region Convolutional 3D Network for Temporal Activity Detection](https://mlsanity.com/paper/1703.07814/) (2017)
- [Learning Salient Boundary Feature for Anchor-free Temporal Action Localization](https://mlsanity.com/paper/2103.13137/) (2021)
- [Gaussian Temporal Awareness Networks for Action Localization](https://mlsanity.com/paper/1909.03877/) (2019)
- [Single Shot Temporal Action Detection](https://mlsanity.com/paper/1710.06236/) (2017)
- [TURN TAP: Temporal Unit Regression Network for Temporal Action Proposals](https://mlsanity.com/paper/1703.06189/) (2017)
- [CDC: Convolutional-De-Convolutional Networks for Precise Temporal Action Localization in Untrimmed Videos](https://mlsanity.com/paper/1703.01515/) (2017)
- [Fast Learning of Temporal Action Proposal via Dense Boundary Generator](https://mlsanity.com/paper/1911.04127/) (2019)
- [A Comprehensive Study of Deep Video Action Recognition](https://mlsanity.com/paper/2012.06567/) (2020)
- [AutoLoc: Weakly-supervised Temporal Action Localization](https://mlsanity.com/paper/1807.08333/) (2018)
- [Background Suppression Network for Weakly-supervised Temporal Action Localization](https://mlsanity.com/paper/1911.09963/) (2019)
- [Active Learning for Deep Detection Neural Networks](https://mlsanity.com/paper/1911.09168/) (2019)
- [Self-Supervised Learning for Semi-Supervised Temporal Action Proposal](https://mlsanity.com/paper/2104.03214/) (2021)
- [Background-Click Supervision for Temporal Action Localization](https://mlsanity.com/paper/2111.12449/) (2021)
- [Learning Temporal Action Proposals With Fewer Labels](https://mlsanity.com/paper/1910.01286/) (2019)
- [Decoupling Localization and Classification in Single Shot Temporal Action Detection](https://mlsanity.com/paper/1904.07442/) (2019)
- [Weakly Supervised Temporal Action Localization with Segment-Level Labels](https://mlsanity.com/paper/2007.01598/) (2020)

## Abstract (arXiv)

Temporal Action Localization (TAL) aims to predict both action category and temporal boundary of action instances in untrimmed videos, i.e., start and end time. Fully-supervised solutions are usually adopted in most existing works, and proven to be effective. One of the practical bottlenecks in these solutions is the large amount of labeled training data required. To reduce expensive human label cost, this paper focuses on a rarely investigated yet practical task named semi-supervised TAL and proposes an effective active learning method, named AL-STAL. We leverage four steps for actively selecting video samples with high informativeness and training the localization model, named Train, Query, Annotate, Append. Two scoring functions that consider the uncertainty of localization model are equipped in AL-STAL, thus facilitating the video sample rank and selection. One takes entropy of predicted label distribution as measure of uncertainty, named Temporal Proposal Entropy (TPE). And the other introduces a new metric based on mutual information between adjacent action proposals and evaluates the informativeness of video samples, named Temporal Context Inconsistency (TCI). To validate the effectiveness of proposed method, we conduct extensive experiments on two benchmark datasets THUMOS'14 and ActivityNet 1.3. Experiment results show that AL-STAL outperforms the existing competitors and achieves satisfying performance compared with fully-supervised learning.